500 Error when trying to visualize datasets saved in `float128` about jupyterlab-h5web HOT 5 CLOSED

Thank you for the report !

Funny that you hit a recursion limit when trying to encode the data. I think it is because the dataset has a dtype we did not account for.

Could you tell us what is the dtype of the dataset you are trying to open ? Or, even better, could you provide us with the problematic file so we can test it out ourselves ? You can scrap other datasets to keep only the problematic one if you wish.

from jupyterlab-h5web.

Hi!
Thank you very much! If you don't mind, is there something you can say for me to further understand this error? I'd love to know what led to this and where the problem lied in further detail!

Yes, I apologize for the delay. I had to go back into my scripts and re-create something that was a lot smaller b/c the files I was looking at were ~76GB. Actually, the h5web extension didn't work with any of them. Here is a link for the file I've just uploaded for you!

Thank you very much!

from jupyterlab-h5web.

Great, thanks for the file : I think I was able to pinpoint the issue.

If you don't mind, is there something you can say for me to further understand this error? I'd love to know what led to this and where the problem lied in further detail!

Sure. As I suspected, this is linked to the data type (or dtype for short) used to save the data.

In your file, the datasets called features are saved using float128 which is not supported by h5web yet. You can check the data type in the Inspect tab, under Type:

Now to solve the issue, we will have to bring support for float128 datasets in jupyterlab-h5web.

However, if you have control over your saving process, the quickest way to solve the problem would be to save the data using float64. ⚠️ Careful though, this will result in loss of numerical precision.

For example, this dataset features_64 (created from your features dataset but converted to float64) can be visualized just fine:

from jupyterlab-h5web.

This will be fixed in next release (landing this week or the next). float128 are now converted to float64 for JSON serialization.

from jupyterlab-h5web.

Understood! Thank you very much!

from jupyterlab-h5web.