What happened of idea using Pure java implementation from jVerbs? about disni HOT 3 CLOSED

Good question. How DiSNI got started was indeed with the pure Java based implementation named jVerbs. The catch, however, was that jVerbs introduced a concept called stateful verbs calls (SVCs). With SVCs, a verbs call like a post send is managed entirely in Java, that is, all the memory used for storing the decscriptor, sg elements, etc., is serialized into Java out-of-heap memory and then written to the hardware directly over memory mapped queues. Later, SVCs turned out to be an efficient way to bypass the JNI boundaries, that is, one can just pass the out-of-heap address of the SVC object to a JNI call and let the C backend interact with the hardware. The overhead of the JNI call is negligible because the call is literally just having one base type parameter which is the address of the SVC object.. With almost no overhead the JNI approach turned out to be more robust and didn't require us to implement vendor specific functionality in Java, like how to access the hardware queue. So over time, the JNI approach replaced the pure Java approach and I don't even know if the pure Java code is stored somewhere still. Does that make sense?

from disni.

from disni.

The bigger problem with the Java only approach is ever changing RDMA user-space libraries and support for all the different libraries. Basically everytime one of the RMDA user libraries (e.g. libmlx or libcxgb) is changed you have to adapt your Java code to reflect the changes. That is not sustainable on the long run. With the current approach we have very little C code in DiSNI mostly wrappers around the libibverbs and librdmacm libraries.

from disni.