Segmentation fault issue possibly due to Keras module in TF2.0 about megnet HOT 5 CLOSED

Hi Nicole @nicolemitchell

Thanks for your interest in our work.

The megnet codebase was written originally using tensorflow 1.x, and was recently upgraded to be compatible with tensorflow 2.0.0 in its main functionalitites. We have not updated all APIs to tensorflow 2.0 yet (e.g., tf.keras and eager mode), since I have noticed issues with tensorflow 2.x in my other works using tensorflow 2.0 during the past months. Notable issues include performance degradation in running in eager mode and also occasional out-of-memory problems.

However, current megnet should not have such problems. We are also using megnet with tensorflow 2.0.0 with no problems. The seg fault problems you described above seem to be related to memory?

For the seg id problems, I am not sure where the problem is. It seems that you are not using molecular data? I have not tested data format other than molecules and crystals, maybe there are some caveats in working with other data. I'd be interested in sorting this out if more details are provided. Do you have a minimal reproducible codebase?

Thanks

Chi

from megnet.

If you are using only part of megnet together with tensorflow2.x APIs and eager mode instead of using keras, it could also result in the problems above.

from megnet.

Hi Chi @chc273

Thanks for your prompt reply!

If you are using only part of megnet together with tensorflow2.x APIs and eager mode instead of using keras, it could also result in the problems above.

I am not using any Tensorflow 2.0 APIs or eager mode. However, I've noticed that the stack trace has calls to Tensorflow's eager mode. Is this expected? Or could it be from how Tensorflow and Keras are configured on my machine?

It seems that you are not using molecular data? I have not tested data format other than molecules and crystals, maybe there are some caveats in working with other data. I'd be interested in sorting this out if more details are provided. Do you have a minimal reproducible codebase?

I am using MEGNet on molecules. As input, I provide the Pybel molecular structures. The target I provide is a vector for each molecule of length num_atoms with binary labels. My model simply modifies the existing MEGNet model to feed only the node embeddings to the final dense layers and return a per-atom classification. Instead of the Set2Set and Concatenation layers following the MEGNet blocks, I create final_vec as follows:

final_vec = Lambda(lambda x: x[0] + 0*x[1][:,0,:])([x1_, x3_])

from megnet.

@nicolemitchell Thanks. This is definitely an interesting topic to explore, but without a minimal example, I cannot really guess what's going on.

Since it involves modifications of the codes and building new models, it seems to be more of a research project instead of an issue of megnet. Currently, we are also exploring possibilities to use megnet in molecular applications.

If you have a working example or a repo, I may be interested to see if I can build off it and include new models to the package.

from megnet.

@nicolemitchell I have updated megnet packages to use tf.keras instead of keras. All tests are passed with tensorflow version 2.1.0

from megnet.