Maximize SFTP performance about libssh2 HOT 6 CLOSED

This isn't really an "issue", you should take discussion items such as this to the mailing list.

1. such a speedup would depend on a lot more factors too than just the buffer size. Latency and bandwidth being two important ones.
2. it doesn't, because it is an internal design thing that we don't expose to apps.
3. I don't think we should. I think we should make the internals as good as possible and remove the need to expose such internal limits.
4. That has been discussed on the list before. We can discuss it further. And it might not be optimal or even perfectly suited for all use cases.
5. I note that you speak of SFTP performance, not other kinds of performance. We've discussed this topic many times before on the mailing list so searching the archives may very well give you more insights.

from libssh2.

Regarding 2.
I added some code that avoids small packets in the following commit: 85dbd4c

So the problem with tiny packets causing slowdowns when the buffer isn't a multiple of MAX_SFTP_READ_SIZE shouldn't happen anymore when reading.

from libssh2.

85dbd4c
Hm, why not simply make "ceil(count / MAX_SFTP_READ_SIZE)" number of request, each requesting "MAX_SFTP_READ_SIZE" bytes? The new code using "size" still has loop holes, e.g. when using buffer_size smaller than MAX_SFTP_READ_SIZE.

from libssh2.

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

from libssh2.

The ticket was automatically closed, but the problem -- libssh2_sftp_write() being slow -- remains, even with the latest 1.10.0 version (I'm using FreeBSD-11 to send).

Not only is the semantics of the function, unfortunately, more like that of write(2), than fwrite(3) -- returning partial writes for bigger buffers instead of looping internally -- the actual throughput is a lot lower than that of the OpenSSH's sftp-client.

A single-threaded uploading to localhost, for example, gets me 11Mb/s at most -- whereas sftp averages about 80Mb/s on the same machine.

Even if the more complex code of non-blocking writing doubles the throughput, it'll still be subpar.

Is this -- SFTP-uploads -- something, the developers are benchmarking regularly?

from libssh2.

IIRC SFTP download implements a pipelined read while write doesn't pipeline uploads which is the speed difference compared to other clients. This can be pipelined, but someone has to do the work. We are always happy to review PRs.

from libssh2.