scsi2nvme
Tools for implementing SCSI on top of NVMe
This repo contains the following:
- SCSI mock device
- NVMe device driver
- Translation Library
- Engine to orchestrate execution flow
Translation Library
The Translation Library offers a two-way command agnostic translation for SCSI commands to NVMe commands and NVMe responses to SCSI responses. It can be used in the user space or the kernel space.
The Library encapsulates away non-caller-actionable issues from the caller. For instance, since the caller is likely to be passing along a pre-constructed SCSI command to the Library, any issues with malformed SCSI commands is not actionable for the caller. In that case, the Library directly writes an error status to the SCSI buffer and no action is needed by the caller.
In cases where the caller is using the Translation Library incorrectly, e.g. calling Translation::Complete before calling Translation::Begin or calling Translation::Begin twice in a row, the library returns an ApiStatus which currently is either kSuccess or kFailure.
Translation Interface
The Translation Interface provides 4 functions:
BeginResponse Begin(Span<const uint8_t> scsi_cmd, Span<const uint8_t> buffer, scsi::LunAddress lun);
Span<const NvmeCmdWrappers> GetNvmeWrappers();
CompleteReponse Complete(Span<const nvme::GenericQueueEntryCpl> cpl_data, Span<const uint8_t> buffer_in, Span<const uint8_t> sense_buffer);
void AbortPipeline();
Translation::Begin()
BeginReponse Begin(Span<const uint8_t> scsi_cmd,
Span<const uint8_t> buffer,
scsi::LunAddress lun)
This function takes in:
- Raw SCSI command in bytes
- Buffer which can either be an input buffer or an output buffer. As an input buffer, it will be where the data from the device is read into. As an output buffer, it stores variable-length command data or the data to be written to the device
- Lun Address which is the NSID in NVMe
From the parameters, the function then delegates to the appropriate command translator and then builds 1 or more NVMe commands. For some commands such as TestUnitReady or Report Supported Op Codes, the Translation Library does not build an NVMe command and instead writes data directly to the SCSI buffer.
The functions returns:
BeginReponse {
ApiStatus status; // Indicates correct usage of Translation Library
uint32_t alloc_len; // Defines size of buffer passed to Translation::Complete
}
Translation::GetNvmeWrappers()
Span<NvmeCmdWrapper> GetNvmeWrappers()
This function returns the constructed NVMe commands along with other useful data.
NvmeCmdWrapper {
nvme::GenericQueueEntryCmd cmd; // NVMe command
uint32_t buffer_len; // length of buffer pointed to by the NVMe PRP pointer
bool is_admin; // whether command is an admin command. Needed for sending to correct NVMe queue
}
Translation::Complete()
CompleteReponse Complete(Span<const nvme::GenericQueueEntryCpl> cpl_data,
Span<const uint8_t> buffer_in,
Span<const uint8_t> sense_buffer);
The function takes in:
- NVMe Completion Queue Entry, which contains the NVMe response data
- SCSI data in buffer where response data is written to
- SCSI sense buffer where status codes are written to
This function translates the NVMe response data and status codes and writes them to the appropriate buffer.
Translation::AbortPipeline
void AbortPipeline()
Finally, in the case that the Translation pipeline needs to be aborted, this function handles all the necesssary memory cleanup.
Intended Usage
- Get the Raw SCSI command and other data from the SCSI subsystem
- Pass data to Translation::Begin()
- Get constructed NVMe commands and other data with Translation::GetNvmeWrappers()
- Send the commands to the NVMe device for execution
- Pass the NVMe completion queue entries to Translation::Complete()
End-to-end translation
Setup
-
Ensure the following:
- an NVMe device is attached to the machine at /dev/nvme0n1
- kernel version is >= 4.19
-
Call
$ makein the root directory. Ensure this results in a scsi2nvme.ko -
Call
$ insmod scsi2nvme.koto insert the kernel module.Upon inserting, the SCSI mock device will be automatically attached. Once the device is attached, the SCSI subsystem will issue a set of commands.
$ lsscsishould show the device with the name "NVMe " (NVMe followed by 4 spaces)
Issuing commands
Issue commands to the SCSI device as such:
Read:
$ sudo dd if=$DEVICE bs=$BLOCK_SIZE count=$NUM_BLOCKS skip=$STARTING_BLOCK_ADDRESS
Write:
$ sudo dd of=$DEVICE bs=$BLOCK_SIZE count=$NUM_BLOCKS seek=$STARTING_BLOCK_ADDRESS
Then you will be prompted to enter the data to be written in stdin. Data can simply be a string of alphanumerical characters such as "The quick brown fox jumps over the lazy dog"
Issue commands to the NVMe device with the nvme-cli Some examples are:
$ sudo nvme read /dev/nvme0n1/-z $BLOCK_SIZE -c $NUM_BLOCKS -s $STARTING_BLOCK_ADDRESS
$ sudo nvme write /dev/nvme0n1/-z $BLOCK_SIZE -c $NUM_BLOCKS -s $STARTING_BLOCK_ADDRESS
See logs
See logs with $ sudo dmesg
Disclaimer
This is not an officially supported Google product.
React
Typescript
Django
Laravel
Facebook
Microsoft
Google
Alibaba
D3
Tencent