AttributeError: partially initialized module 'keras' about nuitka HOT 9 CLOSED

The portpicker function can actually be inlined, I will do that instead of giving up on it, but we can also try to get rid of it too, since it seems to be a test code itself, checking it out now with --noinclude-custom-mode=tensorflow.python.distribute.multi_worker_test_base:error and lets see if that helps, pushing the bloat problem closer to its origin.

from nuitka.

i realized this just now, this is a duplicate of #2751 at least until the keras issue is hit.

from nuitka.

This reproduces it for me too from tensorflow.experimental import dtensor causing this:

  File "C:\Users\kayha\repos\Py2C\MINI3~1.DIS\keras\src\backend\tensorflow\distribution_lib.py", line 10, in <module keras.src.backend.tensorflow.distribution_lib>
ModuleNotFoundError: No module named 'tensorflow.experimental'

I don't see how it doesn't work, indeed something the lazy loader would be supposed to do, but is not happening with tensorflow or keras somehow. The tensorflow.experimental will only be set when the code returns from that import.

from nuitka.

So, I traced this down to where it crashes, and it seems that the tf imported nearby, does have the experimental name. Now I am not yet convinced this can be a import mechanic error, but at first glance it's non-obvious why it wouldn't work in this case, and I will have to look into this.

from nuitka.

I only now noticed, that I am not having the precise error that you got. I am pretty sure I sneaked by those lines, but I will later try your example too. It seems I am loading tensorflow first, and there it doesn't occur. Might be more clever to first sort out that lazy loader, just in case it corrupts tensorflow somehow.

from nuitka.

So, this is self inflicted damage.

      append_plain: |
        import sys
        sys.modules["tensorflow.compat"] = compat
        sys.modules["tensorflow.compat.v1"] = compat.v1
        sys.modules["tensorflow.compat.v2"] = compat.v2
        sys.modules["tensorflow.compat.v1.logging"] = compat.v1.logging
        sys.modules["tensorflow.compat.v2.experimental"] = compat.v2.experimental
        sys.modules["tensorflow.keras"] = keras
        sys.modules["tensorflow.keras.layers"] = keras.layers
        sys.modules["tensorflow.keras.activations"] = keras.activations
        sys.modules["tensorflow.keras.models"] = keras.models

This triggers the lazy loader I think, and then the bad thing happens. There is no line for this:

File "C:\Users\Divine\Desktop\OSSCON~1\NUITKA~1\1242-S~1\MAIN~1.DIS\tensorflow\__init__.py", line 499, in <module tensorflow>

So something in the import design changed there at some time. Now question is of course, what version needed that, and what version doesn't need it anymore, and did it need it for keras ever. These things have critically underdocumented history. I guess, the way forward is to make use see old tensorflow works as is, and then make newest one work, and then bisect to the point where it fails. Given the time a single compilation of these takes, no a quick thing.

from nuitka.

removing your append_plain results in ModuleNotFoundError: No module named 'tensorflow.experimental'
keeping it, still gives the AttributeError: partially initialized module 'keras' has no attribute '__version__' (most likely due to a circular import) on factory

from nuitka.

with the most recent changes, on factory, this example works now.

from nuitka.

Part of the 2.1.4 hotfix release.

from nuitka.