tnewman / dray Goto Github PK

WinSCP displays errors when the home directory is missing. The home directory should be implicitly created.

Switch to Test Containers (https://docs.rs/testcontainers/latest/testcontainers/), so that developers don't need to run Docker Compose in a separate window to run the tests.

Thank you for what you do, but there is a question. Why don't you publish it as a crate? This can be used as a good and simple out-of-the-box sftp server implementation

Add a healthcheck and readiness endpoint (HTTP /health and /ready), so that container orchestrators can determine if the service is alive or not.

Support Google Cloud Storage (https://cloud.google.com/storage?hl=en)

With Windows OpenSSH (C:\Windows\System32\OpenSSH\sftp.exe) , the REALPATH request gets split into 2 packets, so it is not successfully parsed:

0000000a and 1000000001000000012e

0000000a: Length 10
10: Type 16 (SSH2_FXP_REALPATH)
00000001: Id 1
00000001: String length 1
2e: String Data - .

Log:
[2022-12-31T00:02:34Z INFO dray::sftp_session] Received request: Init(Init { version: 0 })
[2022-12-31T00:02:34Z INFO dray::sftp_session] Sending response: Version(Version { version: 3 })
[2022-12-31T00:02:34Z DEBUG dray::protocol::response] Response bytes: 000000050200000003
[2022-12-31T00:02:34Z DEBUG russh::cipher] writing, seqn = 10
[2022-12-31T00:02:34Z DEBUG russh::cipher] padding length 5
[2022-12-31T00:02:34Z DEBUG russh::cipher] packet_length 24
[2022-12-31T00:02:34Z DEBUG russh::cipher] reading, len = [212, 244, 33, 52]
[2022-12-31T00:02:34Z DEBUG russh::cipher] reading, seqn = 10
[2022-12-31T00:02:34Z DEBUG russh::cipher] reading, clear len = 40
[2022-12-31T00:02:34Z DEBUG russh::cipher] read_exact 44
[2022-12-31T00:02:34Z DEBUG russh::cipher] read_exact done
[2022-12-31T00:02:34Z DEBUG russh::cipher] reading, padding_length 10
[2022-12-31T00:02:34Z DEBUG dray::protocol::request] Request bytes: 0000000a
[2022-12-31T00:02:34Z ERROR dray::ssh_server] Received unparsable request!
[2022-12-31T00:02:34Z DEBUG russh::cipher] reading, len = [70, 180, 113, 89]
[2022-12-31T00:02:34Z DEBUG russh::cipher] reading, seqn = 11
[2022-12-31T00:02:34Z DEBUG russh::cipher] reading, clear len = 40
[2022-12-31T00:02:34Z DEBUG russh::cipher] read_exact 44
[2022-12-31T00:02:34Z DEBUG russh::cipher] read_exact done
[2022-12-31T00:02:34Z DEBUG russh::cipher] reading, padding_length 4
[2022-12-31T00:02:34Z DEBUG dray::protocol::request] Request bytes: 1000000001000000012e
[2022-12-31T00:02:34Z ERROR dray::ssh_server] Received unparsable request!

Add an optional test suite to validate multiple SFTP clients on different platforms.

This is a prerequisite for a production ready release.

Support Azure Blob Storage (https://azure.microsoft.com/en-us/products/storage/blobs)

Dray needs a snazzy logo. The logo should be consistent with this definition of a dray: https://www.dictionary.com/browse/dray

When using WinSCP, the client will receive a permission error trying to lstat the home directory.

[2022-12-30T16:33:49Z INFO  dray::sftp_session] Received request: Realpath(Path { id: 3856, path: "/home/tnewman/.." })
[2022-12-30T16:33:49Z INFO  dray::sftp_session] Sending response: Name(Name { id: 3856, files: [File { file_name: "/home", file_attributes: FileAttributes { size: None, uid: None, gid: None, permissions: Some(16895), atime: None, mtime: None } }] })
[2022-12-30T16:33:49Z INFO  dray::sftp_session] Received request: Lstat(Path { id: 4103, path: "/home" })
[2022-12-30T16:33:49Z ERROR dray::sftp_session] Received error while processing request: Permission denied.
[2022-12-30T16:33:49Z INFO  dray::sftp_session] Sending response: Status(Status { id: 4103, status_code: PermissionDenied, error_message: "The client has insufficient privileges to perform the requested operation." })

Switch from Docker to ContainerD (nerdctl)

See #6

• Binaries for common platforms
• Docker images
• Helm charts
• EC2 AMIs?

Knowing what went wrong when errors occur is somewhat difficult right now. Ideally, we would have logging and tracing to be able to easily correlate logs.

The test suite currently does not run on Windows (and possibly other platforms) because it shells out to the sftp command and assumes a certain set of functionality.

1. Update the test suite to use a Rust library for sftp instead of shelling out. https://github.com/veeso/remotefs-rs is an example.
2. Update the GitHub Actions build matrix to build on Windows and Mac OSX

On SIGINT:

1. Stop accepting new connections
2. Allow existing connections to finish their work

Graceful shutdown is very important to make sure clients are not interrupted during scale-in, maintenance, etc.

Since SFTP is connection oriented, there will need to be a limit to how long to wait.

Test Containers are not cleaned up because they are static, so drop is never run.

Creating a separate container for each test results in intermittent failures.

One idea: Get rid of Test Containers and switch back to Docker Compose. Each test will create it's own directory to provide isolation. The directory will get pruned at the end of the test. The Dray server can also be shared for all tests to avoid spawning a separate server for each test.

Switch to the official AWS SDK for Rust: https://aws.amazon.com/sdk-for-rust/