Git Product home page Git Product logo

go-tcmu's Introduction

go-tcmu

GoDoc

Go bindings to attach Go Readers and Writers to the Linux kernel via SCSI.

It connects to the TCM Userspace kernel API, and provides a loopback device that responds to SCSI commands. This project is based on open-iscsi/tcmu-runner, but in pure Go.

Overview

This package creates two types of Handlers (much like net/http) for SCSI block device commands. It wraps the implementation details of the kernel API, and sets up (a) a TCMU SCSI device and connect that to (b) a loopback SCSI target.

From here, the Linux IO Target kernel stack can expose the SCSI target however it likes. This includes iSCSI, vHost, etc. For further details, see the LIO wiki.

Usage

First, to use this package, you'll need the appropriate kernel modules and configfs mounted

Make sure configfs is mounted

This may already be true on your system, depending on kernel configuration. Many distributions do this by default. Check if it's mounted to /sys/kernel/config with

mount | grep configfs

Which should respond

configfs on /sys/kernel/config type configfs (rw,relatime)

To mount it explicitly:

sudo modprobe configfs
sudo mkdir -p /sys/kernel/config
sudo mount -t configfs none /sys/kernel/config

Use the TCMU module

Many distros include the module, but few activate it by default.

sudo modprobe target_core_user

Now that that's settled, there's tcmufile.go for a quick example binary that serves an image file under /dev/tcmufile/myfile.

For creating your custom SCSI targets based on a ReadWriterAt:

handler := &tcmu.SCSIHandler{
        HBA: 30, // Choose a virtual HBA number. 30 is fine.
        LUN: 0,  // The LUN attached to this HBA. Multiple LUNs can work on the same HBA, this differentiates them.
        WWN: tcmu.NaaWWN{
                OUI:      "000000",                      // Or provide your OUI
                VendorID: tcmu.GenerateSerial("foobar"), // Or provide a vendor id/serial number
                // Optional: Provide further information for your WWN
                // VendorIDExt: "0123456789abcdef", 
        },
        VolumeName: "myVolName", // The name of your volume.
        DataSizes: tcmu.DataSizes{
                VolumeSize: 5 * 1024 * 1024, // Size in bytes, eg, 5GiB
                BlockSize:  1024,            // Size of logical blocks, eg, 1K
        },
        DevReady: tcmu.SingleThreadedDevReady(
                tcmu.ReadWriterAtCmdHandler{      // Or replace with your own handler
                        RW: rw,
                }),
}
d, _ := tcmu.OpenTCMUDevice("/dev/myDevDirectory", handler)
defer d.Close()

This will create a device named /dev/myDevDirectory/myVolName with the mentioned details. It is now ready for formatting and treating like a block device.

If you wish to handle more SCSI commands, you can implement a replacement for the ReadWriterAtCmdHandler following the interface:

type SCSICmdHandler interface {
	HandleCommand(cmd *SCSICmd) (SCSIResponse, error)
}

If the default functionality was acceptable, the library contains a number of helpful Emulate functions that you can call to achieve the basic functionality.

go-tcmu's People

Contributors

barakmich avatar ericchiang avatar jonboulle avatar nak3 avatar vbatts avatar xiang90 avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

go-tcmu's Issues

When some program on the same host enabled netlink support for target_core_user.ko, go-tcmu program hangs

description

When some program that uses target_core_user.ko enables netlink support (TCMU_ATTR_SUPP_KERN_CMD_REPLY), go-tcmu hangs.

reproduce steps

  1. Run tcmu-runner (to enable netlink reply support).
  # ./tcmu-runner
  // You can stop tcmu-runner process as it keeps the netlink support enabled //
  1. Start go-tcmu program
  # go run cmd/tcmufile/tcmufile.go /tmp/foo
  ... snip ...
  DEBU[0000] Setting /sys/kernel/config/target/core/user_30/foo/control: dev_config=go-tcmu//foo
  DEBU[0000] Setting /sys/kernel/config/target/core/user_30/foo/control: hw_block_size=1024
  DEBU[0000] Setting /sys/kernel/config/target/core/user_30/foo/control: async=1
  DEBU[0000] Setting /sys/kernel/config/target/core/user_30/foo/enable: 1
  //  hang forever //

address control file not exists / kernel 4.4.32

I'm trying to run go-tcmu with custom handler on linux 4.4 and found that there is no address control file in configfs.

DEBU[0000] Got a TCMU mailbox, version: 2
           source="device.go:310"
DEBU[0000] mapsize: 1118208
           source="device.go:311"
DEBU[0000] mbFlags: 0
           source="device.go:312"
DEBU[0000] mbCmdrOffset: 128
           source="device.go:313"
DEBU[0000] mbCmdrSize: 65408
           source="device.go:314"
DEBU[0000] mbCmdHead: 0
           source="device.go:315"
DEBU[0000] mbCmdTail: 0
           source="device.go:316"
DEBU[0000] SCSI EVPD Inquiry 0x0
           source="cmd_handler.go:103"
DEBU[0000] SCSI EVPD Inquiry 0x83
           source="cmd_handler.go:103"
FATA[0000] Couldn't create tcmu in /dev/xxx: open /sys/kernel/config/target/loopback/naa.50000000098f6bcd/tpgt_1/address: no such file or directory

After ~30s kernel hangs:

Nov 23 02:22:25 pool03e kernel: [ 2224.774431] NMI watchdog: BUG: hard lockup - CPU#10 stuck for 10s! [swapper/10:0] at native_queued_spin_lock_slowpath+0x121/0x170
Nov 23 02:22:25 pool03e kernel: [ 2224.774490] Modules linked in: tcm_loop target_core_pscsi target_core_file target_core_iblock target_core_user target_core_mod uio ipt_MASQUERADE nf_nat_masquerade_ipv4 bridge iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat xt_addrtype nf_conntrack_ipv6 nf_defrag_ipv6 xt_u32 xt_tcpudp xt_comment xt_multiport xt_conntrack nf_conntrack veth overlay quota_v2 quota_tree tcp_diag inet_diag unix_diag ip6table_filter ip6_tables iptable_filter ip_tables x_tables cls_cgroup sch_htb netconsole configfs dummy 8021q mrp garp stp llc intel_powerclamp coretemp kvm_intel gpio_ich joydev input_leds mac_hid kvm lpc_ich 8250_fintek tpm_infineon i5500_temp serio_raw shpchp irqbypass crc32_pclmul i7core_edac ioatdma edac_core mlx4_en mlx4_core vxlan udp_tunnel ip6_udp_tunnel tcp_htcp hid_generic usbhid hid raid10 raid456 async_raid6_recov async_memcpy async_pq async_xor async_tx xor raid6_pq libcrc32c raid1 igb raid0 i2c_algo_bit dca ahci multipath ptp psmouse libahci pps_core linear fjes [last unloaded: pmi_msghandler]
Nov 23 02:22:25 pool03e kernel: [ 2224.774543] CPU: 10 PID: 0 Comm: swapper/10 Not tainted 4.4.32-11 #1
Nov 23 02:22:25 pool03e kernel: [ 2224.774544] Hardware name: Supermicro X8DTU/X8DTU, BIOS 2.1c       08/03/2012
Nov 23 02:22:25 pool03e kernel: [ 2224.774545] task: ffff881808ede200 ti: ffff880c08d90000 task.ti: ffff880c08d90000
Nov 23 02:22:25 pool03e kernel: [ 2224.774547] RIP: 0010:[<ffffffff810f4a01>]  [<ffffffff810f4a01>] native_queued_spin_lock_slowpath+0x121/0x170
Nov 23 02:22:25 pool03e kernel: [ 2224.774550] RSP: 0018:ffff88180f303d20  EFLAGS: 00000046
Nov 23 02:22:25 pool03e kernel: [ 2224.774551] RAX: 0000000000000000 RBX: 0000000000000002 RCX: ffff88180f317380
Nov 23 02:22:25 pool03e kernel: [ 2224.774552] RDX: ffffffff81f50a78 RSI: 00000000002c0000 RDI: ffff881725e38500
Nov 23 02:22:25 pool03e kernel: [ 2224.774553] RBP: ffff88180f303d20 R08: 0000000000000101 R09: 0000000000000000
Nov 23 02:22:25 pool03e kernel: [ 2224.774555] R10: 0000000000000658 R11: 0000000000000005 R12: ffff881725e38500
Nov 23 02:22:25 pool03e kernel: [ 2224.774556] R13: 0000000000000000 R14: 0000000000000000 R15: 00000000ffffffff
Nov 23 02:22:25 pool03e kernel: [ 2224.774557] FS:  0000000000000000(0000) GS:ffff88180f300000(0000) knlGS:0000000000000000
Nov 23 02:22:25 pool03e kernel: [ 2224.774558] CS:  0010 DS: 0000 ES: 0000 CR0: 000000008005003b
Nov 23 02:22:25 pool03e kernel: [ 2224.774559] CR2: 00007f5bb9ffafb8 CR3: 0000000001e0b000 CR4: 00000000000006e0
Nov 23 02:22:25 pool03e kernel: [ 2224.774561] Stack:
Nov 23 02:22:25 pool03e kernel: [ 2224.774562]  ffff88180f303d38 ffffffff8180eeae ffff881725e383f0 ffff88180f303d88
Nov 23 02:22:25 pool03e kernel: [ 2224.774564]  ffffffffa045ac74 ffffffff82130ac2 ffff880b00000022 0000000000000286
Nov 23 02:22:25 pool03e kernel: [ 2224.774566]  ffff881725e38660 ffff88180f303dd8 0000000000000000 00000000000000ff
Nov 23 02:22:25 pool03e kernel: [ 2224.774568] Call Trace:
Nov 23 02:22:25 pool03e kernel: [ 2224.774569]  <IRQ> 
Nov 23 02:22:25 pool03e kernel: [ 2224.774576]  [<ffffffff8180eeae>] _raw_spin_lock_irqsave+0x3e/0x50
Nov 23 02:22:25 pool03e kernel: [ 2224.774596]  [<ffffffffa045ac74>] target_complete_cmd+0x44/0x1f0 [target_core_mod]
Nov 23 02:22:25 pool03e kernel: [ 2224.774599]  [<ffffffffa04aeba1>] tcmu_check_expired_cmd+0x41/0x60 [target_core_user]
Nov 23 02:22:25 pool03e kernel: [ 2224.774601]  [<ffffffffa04af280>] ? tcmu_irqcontrol+0x20/0x20 [target_core_user]
Nov 23 02:22:25 pool03e kernel: [ 2224.774605]  [<ffffffff813c800a>] idr_for_each+0x9a/0xe0
Nov 23 02:22:25 pool03e kernel: [ 2224.774607]  [<ffffffffa04aeb60>] ? tcmu_open+0x40/0x40 [target_core_user]
Nov 23 02:22:25 pool03e kernel: [ 2224.774610]  [<ffffffffa04af2d8>] tcmu_device_timedout+0x58/0x80 [target_core_user]
Nov 23 02:22:25 pool03e kernel: [ 2224.774612]  [<ffffffffa04af280>] ? tcmu_irqcontrol+0x20/0x20 [target_core_user]
Nov 23 02:22:25 pool03e kernel: [ 2224.774615]  [<ffffffff8110e102>] call_timer_fn+0x42/0x160
Nov 23 02:22:25 pool03e kernel: [ 2224.774617]  [<ffffffff8110d90a>] ? cascade+0x4a/0x80
Nov 23 02:22:25 pool03e kernel: [ 2224.774618]  [<ffffffff8110f801>] run_timer_softirq+0x281/0x2d0
Nov 23 02:22:25 pool03e kernel: [ 2224.774621]  [<ffffffffa04af280>] ? tcmu_irqcontrol+0x20/0x20 [target_core_user]
Nov 23 02:22:25 pool03e kernel: [ 2224.774624]  [<ffffffff810b2714>] __do_softirq+0xd4/0x2c0
Nov 23 02:22:25 pool03e kernel: [ 2224.774625]  [<ffffffff810b2b2b>] irq_exit+0x7b/0xa0
Nov 23 02:22:25 pool03e kernel: [ 2224.774629]  [<ffffffff81811b86>] smp_apic_timer_interrupt+0x46/0x60
Nov 23 02:22:25 pool03e kernel: [ 2224.774631]  [<ffffffff8180fe22>] apic_timer_interrupt+0x82/0x90

I've searched thru open-iscsi/tcmu-runner and Documentation/target/tcmu-design.txt and didn't found any references to this control file. Seems that they're using another kernel API to add new devices (UIO as far as I understand).

Am I running incompatible kernel version? Which minimum version do I need to get go-tcmu working?

Thanks,
Gleb.

command handler doesn't output the read and write errors

Current codes check the result of Read or Write like this:

  n, err := r.ReadAt(cmd.Buf[:length], int64(offset))
	if n < length {
		log.Errorln("read/read failed: unable to copy enough")
		return cmd.MediumError(), nil
	}
	if err != nil {
		log.Errorln("read/read failed: error:", err)
		return cmd.MediumError(), nil
	}

When Read() and Write() return error, they always make n < length true, and so they return before outputting the error.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.