Hello @mano3-1!
Thank you to share your code, it's a really interesting application.
I'm trying to reproduce your process, but I'm having this error in colab, when function model.fit() is called inside the execution of --> test = train(fnames ,dict_ ,model)

ValueError: Failed to convert a NumPy array to a Tensor (Unsupported object type numpy.ndarray).

I have tried to add .astype('object') or .astype('float32') when load the values into an array previous to train, but without success. Dimensions of the X ndarray are correct, as you can see here, but i still have the problem:

print(type(X)) --> <class 'numpy.ndarray'>
print(len(X)) #folders --> 51
print(len(X[0])) #frame --> 20
print(len(X[0][0])) #heatmap -->16
print(len(X[0][0][0])) #heatmap --> 21
print(len(X[0][0][0][0])) #heatmap --> 17
print(X.size) --> 1020 # 51 train folders * 20 frames

Did you have this problem before?
Thanks in advance!
nice work,
Mario

Extra info:
I'm working on pose_detection.ipynb notebook after generating succesfully the folders with .npy files of classes "Punch" and "WalkingWithDog"