Can't figure out interesting behavior by Matrix4f.transpose() about joml HOT 1 CLOSED

Thanks for reporting!
And, sorry about that. It is a bug about the Matrix4f.set(col, row, value) method not resetting the assumptions about the matrix properties, that are stored in the properties field in order to route/accelerate further operations.
It has been fixed for 1.10.6, which you can use right now with 1.10.6-SNAPSHOT or in the actual 1.10.6 release later.

Generally, manually setting any matrix field is highly discouraged, because it will break optimized matrix operations afterwards, which then always have to make no assumptions anymore and use the most general operation implementations.

In order to "reestablish" the properties of a matrix, there is the method determineProperties(), but this is not able to determine all properties of a matrix, such as orthogonality. So, this can be called after setting all matrix fields manually, to get back at least some performance for further operations.

from joml.