targetcli

The Linux SCSI Target Wiki

(Difference between revisions)
Jump to: navigation, search
m (→Startup)
(Add bit about how to mount configfs)
Line 65: Line 65:
{{Ambox| type=info| head=[[{{OS}}]] Admin Manual| text=The {{OS Admin Manual}} provides comprehensive background and many examples on using ''targetcli'' and on programming the target library.}}
{{Ambox| type=info| head=[[{{OS}}]] Admin Manual| text=The {{OS Admin Manual}} provides comprehensive background and many examples on using ''targetcli'' and on programming the target library.}}
 +
 +
=== Mounting configFS ===
 +
 +
{{T}} and targetcli uses [[configFS]] for its configuration. All storage objects can be accessed and exported, so the configFS module must be loaded and configFS must be mounted. Mount configFS at <code>/sys/kernel/config</code>:
 +
 +
<pre>
 +
mount -t configfs configfs /sys/kernel/config
 +
</pre>
 +
 +
This can also be added to <code>/etc/fstab</code> to automatically mount at boot:
 +
 +
<pre>
 +
configfs /sys/kernel/config configfs defaults 1 1
 +
</pre>
 +
 +
Note that /etc/init.d/target will automatically mount [[configFS]] if it's not already been mounted by /etc/fstab or manually by user.
=== Startup ===
=== Startup ===
Line 86: Line 102:
</pre>
</pre>
-
Upon ''targetcli'' initialization, the underlying RTSlib loads the installed fabric modules, and creates the corresponding [[ConfigFS]] mount points (at ''/sys/kernel/config/target/<fabric>''), as specified by the associated spec files (located in ''/var/target/fabric/fabric.spec'').
+
Upon ''targetcli'' initialization, the underlying RTSlib loads the installed fabric modules, as specified by the associated spec files (located in ''/var/target/fabric/fabric.spec'').
=== Display help ===
=== Display help ===

Revision as of 18:24, 17 April 2014

targetcli
Logo
LIO 150513.png
Storage Manager
Original author(s) Jerome Martin
Michael Kromer
Nicholas Bellinger
Developer(s) Datera, Inc.
Initial release October 14, 2009 (2009-10-14)
Stable release 2.00 / March 1, 2012;
9 years ago (2012-03-01)
Development status Production
Written in Python
Operating system Linux
Type Storage management
License AGPLv3
Website datera.io
targetcli running with mixed-mode iSCSI, SCSI virtualziation, FCoE, Fibre Channel and IB/SRP.

targetcli is the single-node LinuxIO management shell by Datera, Inc..

Contents

Overview

targetcli is the general management platform for the LinuxIO. It comprises a shell that uses the LIO library through a well-defined API, and is available under the Apache License, version 2.0 (Apache License). Thus targetcli can easily be replaced or complemented by a UI with other metaphors, such as a GUI, to provide the same functionality by different means. Everyone is welcome to contribute.

targetcli was released on October 14, 2009 (2009-10-14), and supports all fabric modules, including FCoE, Fibre Channel, IBM vSCSI, iSCSI, iSER, SRP, tcm_loop, and vHost. It is is based on a modular, extensible architecture, with plug-in modules for additional functionality.

targetcli consists of three Python modules:

Asymmetric Logical Unit Assignment (ALUA)
Full Asymmetric Logical Unit Assignment ALUA configuration still requires lio-utils for the time being.

Download and build instructions for targetcli are on the download page.

Linux distributions

targetcli and LIO are included in most Linux distributions per default. Here is an overview over the most popular distributions:

Distribution Version[Linux 1] Release Archive Install Source git[Linux 2] Documentation
CentOS 6.2 2011-12-20 CentOS mirror su -c 'yum install fcoe-target-utils' targetcli-fb.git Tech Notes
Debian 7.0 ("wheezy") TBA Debian pool su -c 'apt-get install targetcli' targetcli
Fedora 16, 17/18 2011-11-08 Fedora Rawhide su -c 'yum install targetcli' targetcli-fb.git Target Wiki
openSUSE 12.1 2011-11-08 Requires manual installation from targetcli.
RHEL 6.2 2011-11-16 Fedora Rawhide su -c 'yum install fcoe-target-utils' targetcli-fb.git Tech Notes
Scientific Linux 6.2 2012-02-16 SL Mirror su -c 'yum install fcoe-target-utils' targetcli-fb.git Tech Notes
SLES SP2 2012-02-15 Requires manual installation from targetcli.
Ubuntu PrecisePangolin v12 2012-04-26 Ubuntu universe su -c 'apt-get install targetcli' targetcli
  1. ↑ The distribution release where LIO was included first.
  2. ↑ Technical support, and qualified backports to other kernels and distributions are available from Datera.

targetcli also has a complementary open-source SCSI test suite (scsi-testuite).

LIO is also natively supported in OpenStack (setup, code) via rtslib, starting with the Grizzly release.

Download and build instructions for targetcli are on the download page.

Quick start guide

The quick start guide provides simple examples of basic target setups that export one block device residing in a file backstore (FILEIO).

targetcli can creates a comprehensive configuration of storage objects with only a few commands by using smart default settings that auto-complete most of the work. targetcli also supports command and parameter completion via <TAB>, so all available commands and parameters can be listed from all contexts.

LIO Admin Manual
The Template:OS Admin Manual provides comprehensive background and many examples on using targetcli and on programming the target library.

Mounting configFS

LIO and targetcli uses configFS for its configuration. All storage objects can be accessed and exported, so the configFS module must be loaded and configFS must be mounted. Mount configFS at /sys/kernel/config: 

mount -t configfs configfs /sys/kernel/config

This can also be added to /etc/fstab to automatically mount at boot: 

configfs /sys/kernel/config configfs defaults 1 1

Note that /etc/init.d/target will automatically mount configFS if it's not already been mounted by /etc/fstab or manually by user.

Startup

Invoke targetcli as root from the command prompt of the underlying OS shell: 

# targetcli
Welcome to targetcli:

 Copyright (c) 2013 by Datera, Inc.
 All rights reserved.

Visit us at http://www.daterainc.com.

Using ib_srpt fabric module.
Using qla2xxx fabric module.
Using iscsi fabric module.
Using loopback fabric module.
/>

Upon targetcli initialization, the underlying RTSlib loads the installed fabric modules, as specified by the associated spec files (located in /var/target/fabric/fabric.spec).

Display help

Use help to get context-sensitive help, depending on the current or selected object context: 

/> help
GENERALITIES
  This is an interactive shell in which you can create, delete and configure
  configuration objects.

  [...]

COMMAND SYNTAX
  Commands are built using the following syntax:

  [TARGET_PATH] COMMAND_NAME [OPTIONS]

  [...]

AVAILABLE COMMANDS
  The following commands are available in the current path:

    - bookmarks action [bookmark]
    - cd [path]
    - create [wwn]
    - delete wwn
    - exit
    - get [group] [parameter...]
    - help [topic]
    - info
    - ls [path] [depth]
    - pwd
    - refresh
    - set [group] [parameter=value...]
    - status
    - version
/> backstores/iblock help create
SYNTAX
  create name dev [generate_wwn]

DESCRIPTION
  Creates an IBlock Storage object. dev is the path to the TYPE_DISK block
  device to use and the optional generate_wwn parameter is a boolean specifying
  whether or not we should generate a T10 wwn Serial for the unit (by default,
  yes).
/> iscsi/ help create
SYNTAX
  create [wwn]

DESCRIPTION
  Creates a new target. The wwn format depends on the transport(s) supported by
  the fabric module. If the wwn is ommited, then a target will be created using
  either a randomly generated WWN of the proper type, or the first unused WWN in
  the list of possible WWNs if one is available. If WWNs are constrained to a
  list (i.e. for hardware targets addresses) and all WWNs are in use, the target
  creation will fail. Use the info command to get more information abour WWN
  type and possible values.

SEE ALSO
  info
/>

Navigate the object tree

targetcli groups the target stack objects into an hierarchical tree of context objects, and allows navigating them correspondingly. Context objects are named by their full path.

cd navigates the object tree. Without parameters, cd presents the full objects tree. The destination path can also be selected via cursor keys.

pwd displays the complete current path (the prompt may display an abbreviated path).

The global parameter auto_cd_after_create controls if targetcli automatically enters new object contexts after the creation of the corresponding new object.

auto_cd_after_create = false
The following examples assume that auto_cd_after_create is set to false. 
/> set global auto_cd_after_create=false
Parameter auto_cd_after_create is now 'false'.
/>

Display the object tree

Use ls to list the object hierarchy, which is initially empty: 

/> ls
o- / ..................................................................... [...]
  o- backstores .......................................................... [...]
  | o- fileio ............................................... [0 Storage Object]
  | o- iblock ............................................... [0 Storage Object]
  | o- pscsi ................................................ [0 Storage Object]
  | o- rd_dr ................................................ [0 Storage Object]
  | o- rd_mcp ............................................... [0 Storage Object]
  o- ib_srpt ........................................................ [0 Target]
  o- iscsi .......................................................... [0 Target]
  o- loopback ....................................................... [0 Target]
  o- qla2xxx ........................................................ [0 Target]
/>

Create a backstore

Enter the top-level backstore object, and create storage objects using IBLOCK, FILEIO or PSCSI type devices.

IBLOCK

Create an IBLOCK backstore, which can be any TYPE_DISK block-device, including LVM logical volumes and /dev/disk/by-id/ symlinks.

Recommended for performance
IBLOCK backstores usually provide the best performance.

For instance, use the /dev/sdb block device: 

/> cd backstores/
/backstores> iblock/ create name=block_backend dev=/dev/sdb
Generating a wwn serial.
Created iblock storage object block_backend using /dev/sdb.
/backstores>

Alternatively, any LVM logical volume can be used as a backstore, please refer to the Template:OS Admin Manual on how to create them properly.

For instance, create an IBLOCK backstore on a logical volume (under /dev/<volume_group_name>/<logical_volume_name>): 

/backstores> iblock/ create name=block_backend_lvm lvm dev=/dev/vg0/lv1
Generating a wwn serial.
Created iblock storage object block_backend_lvm using /dev/vg0/lv1.
/backstores>

In both cases, targetcli automatically creates a WWN serial ID for the new backstore devices.

FILEIO

Create an FILEIO backstore with the underlying file /usr/src/fileio and size of 2 GB: 

/> cd backstores/
/backstores> fileio/ create name=file_backend file_or_dev=/usr/src/fileio size=2G
Generating a wwn serial.
Not using buffered mode.
Created fileio file_backend.
/backstores>

targetcli automatically creates a WWN serial ID for the backstore device.

By default, FILEIO uses non-buffered mode (i.e. O_SYNC is set).

Do not use buffered FILEIO
Creating a FILEIO backend with buffered=True enables the buffer cache. While this can provide a significant performance increase, it also creates a serious data integrity hazard: If the system crashes for any reason, an unflushed buffer cache can cause the entire backstore to be irrecoverably corrupted. Use RTS OS with robust flash caching instead.

PSCSI

Create a PSCSI (SCSI pass-through) backstore for a physical SCSI device, in this case a TYPE_ROM device using /dev/sr0: 

/backstores> pscsi/ create name=pscsi_backend dev=/dev/sr0
Generating a wwn serial.
Created pscsi storage object pscsi_backend using /dev/sr0.
/backstores>

targetcli automatically creates a WWN serial ID for the backstore device.

Do not use PSCSI
Do not use PSCSI unless you know exactly how it will be used. Advanced SCSI CDBs such as for Persistent Reservations or ALUAs (used e.g. by VMware ESX and vSphere) are typically not implemented in the device firmware, and can cause malfunctions or crashes. Use IBLOCK for production setups instead.

Ramdisk

Create a RAM disk backstore with a size of 1 GB: 

/backstores> rd_mcp/ create name=rd_backend size=1GB
Generating a wwn serial.
Created rd_mcp ramdisk rd_backend with size 1GB.
/backstores>

targetcli automatically creates a WWN serial ID for the backstore device.

Display the resulting object tree

Use ls to list the resulting object hierarchy (displayed from the root object): 

/> ls
o- / ..................................................................... [...]
  o- backstores .......................................................... [...]
  | o- fileio ............................................... [1 Storage Object]
  |   o- file_backend ............................ [/usr/src/fileio deactivated]
  | o- iblock .............................................. [2 Storage Objects]
  | | o- block_backend .................................. [/dev/sdb deactivated]
  | | o- block_backend_lvm .......................... [/dev/vg0/lv1 deactivated]
  | o- pscsi ................................................ [1 Storage Object]
  |   o- pscsi_backend .................................. [/dev/sr0 deactivated]
  | o- rd_dr ................................................ [0 Storage Object]
  | o- rd_mcp ............................................... [1 Storage Object]
  |   o- rd_backend ...................................... [ramdisk deactivated]
  o- ib_srpt ........................................................ [0 Target]
  o- iscsi .......................................................... [0 Target]
  o- loopback ....................................................... [0 Target]
  o- qla2xxx ........................................................ [0 Target]
/>

Create a target

LinuxIOs can now be instantiated as described for the fabrics, respectively:

Persist the configuration

Don't forget to use saveconfig!
Without saveconfig, the target configuration will be lost upon rebooting or unloading the target service, as the target configuration will roll back to the last saved configuration.

The target configuration can be persisted across reboots by using saveconfig from the root context: 

/> saveconfig
WARNING: Saving rtsnode1 current configuration to disk will overwrite your boot settings.
The current target configuration will become the default boot config.
Are you sure? Type 'yes': yes
Making backup of srpt/ConfigFS with timestamp: 2012-02-27_23:19:37.660264
Successfully updated default config /etc/target/srpt_start.sh
Making backup of qla2xxx/ConfigFS with timestamp: 2012-02-27_23:19:37.660264
Successfully updated default config /etc/target/qla2xxx_start.sh
Making backup of loopback/ConfigFS with timestamp: 2012-02-27_23:19:37.660264
Successfully updated default config /etc/target/loopback_start.sh
Making backup of LIO-Target/ConfigFS with timestamp: 2012-02-27_23:19:37.660264
Successfully updated default config /etc/target/lio_backup-2012-02-27_23:19:37.660264.sh
Making backup of Target_Core_Mod/ConfigFS with timestamp: 2012-02-27_23:19:37.660264
Successfully updated default config /etc/target/tcm_backup-2012-02-27_23:19:37.660264.sh
Generated Target_Core_Mod config: /etc/target/backup/tcm_backup-2012-02-27_23:19:37.660264.sh
Successfully updated default config /etc/target/lio_start.sh
Successfully updated default config /etc/target/tcm_start.sh
/> exit
#

Basic concepts

There are two complementary key concepts that are fundamental to understanding how to use targetcli:

General

The fundamental metaphor of targetcli are context objects, which represent the target stack objects. Context objects are unified in a single hierarchical object tree that reflects their logical structure and relations. Context objects are named by their full path in the hierarchical object tree, which allows addressing and navigating them.

'Entering' an object changes the current object context, corresponding to its current working path, which is depicted in the command prompt.

pwd alternatively displays the complete current path (e.g., if the prompt displays an abbreviated path for space efficiency).

cd navigates the object tree. Without parameters, cd presents the full objects tree. The destination path can be selected via cursor keys.

help cd lists context-sensitive navigation tips.

Due to the hierarchical structure of an LIO object tree, multiple context changes might be required to reach and enter a particular object. For instance:

Working with contexts

create context <context_name>, or short #<context_name>, saves the current context under 'name', which simplifies traversal of the object tree. Saved contexts can be restored at any time and remain persistent across sessions.

enter context <context_name>, or short @<context_name>, restores the corresponding saved context 'name' and immediately enters it. Saved contexts allow both naming and bookmarking available transport objects.

Each context objects provides context-sensitive operations, i.e. different context objects (or paths) provide different command sets. For instnace, a path pointing at an iSCSI target provides different commands than a path pointing at a storage object.

Command completion

TAB completion
At any time in the shell, <TAB> triggers command auto-completion. You can use it to display all available commands and options, depending on the current context.

With a unique option, pressing <TAB> once auto-completes the current command being edited. With multiple options, hitting <TAB> twice produces a list of available command completions, if any, and a quick syntax help on the command currently being edited. This is useful when the exact next parameters available for the current command are not known.

Each command parameter can be passed either as a positional parameter, following the order of the command syntax, or as a key=value pair, in any order. Command auto-completion will reflect and present all available options.

help followed by any command name lists the full command syntax and description.

Command syntax

Use <verb> on <object type> and <object specifiers>. 

[<verb>] <object_type> [<object specifier>...]

Example: create target creates a target for a 'transport' (FCoE, Fibre Channel, iSCSI, iSER, SRP, etc.) in the root context. If the verb is omitted, the default is to use the first available implicit action verb for the object, in that order: enter, list and get. 

<command> [<argument>] [<argument>]

Apart from generic verbs, there are also global commands, like help or exit. They take optional arguments and do not work on transport context objects or info objects.

Command chains

targetcli has the ability to chain commands, which provides powerful semantics for creating complex command sequences. Command chains are constructed by chaining multiple single commands together, separated by a comma: 

command1, command2 [,command...]

When a command results in a context change (i.e. enter target iqn.1999-03.org.foobar:1234 enters that target's context), the next command in the chain executes in that new context. In addition, create context and enter context can be used in command chains, enabling easy scripting of creation, deletion and query operations of objects embedded deeply in the LIO object tree.

Global parameters

There are numerous global and context-sensitive parameters that modify the behavior of targetcli.

Here are two particularly important ones:

Object tree

The LIO object hierarchy encompasses both the fabric modules (FCoE, Fibre Channel, iSCSI, iSER, SRP, etc.) and the low-level objects used to map the underlying storage objects.

Fabric objects

Here is a summary of the iSCSI fabric objects hierarchy (see also the underlying configFS layout): 

+-targetcli
  |
  +-Targets
    | Identified by their WWNs or IQN (for iSCSI).
    | Targets identify a group of Endpoints.
    |
    +-TPGs (Target Portal Groups, iSCSI only)
      | The TPG is identified by its numerical Tag, starting at 1. It
      | groups several Network Portals, and caps LUNs and Node ACLs.
      | For fabrics other than iSCSI, targetcli masks the TPG level.
      |
      +-Network Portals (iSCSI only)
      |   A Network Portal adds an IP address and a port. Without at
      |   least one Network Portal, the Target remains disabled.
      |
      +-LUNs
      |   LUNs point at the Storage Objects, and are numbered 0-255.
      |
      +-ACLs
        | Identified by initiator WWNs/IQNs, ACLs group permissions
        | for that specific initiator. If ACLs are enabled, one
        | NodeACL is required per authorized initiator.
        |
        + Mapped LUNs
            Determine which LUNs an initiator will see. E.g., if
            Mapped LUN 1 points at LUN 0, the initiator referenced
            by the NodeACL will see LUN 0 as LUN 1.

Storage objects

Here is a summary of the LIO storage object hierarchy: 

+-Root
  |
  +-VirtualHBAs
    | These are the emulated Host Bus Adapters of the TCM layer.
    | They are identified by their HBA number and their backstore type:
    |   . iblock - emulates SCSI on top of any TYPE_DISK block device.
    |   . fileio - emulates SCSI on top of the local filessystem.
    |   . rd_mcp - ramdisk drive-based SCSI emulation, a bit slower
    |              ramdisk, but more robust with multiple initiators,
    |              with a separate memory mapping using memory copy.
    |   . rd_dr  - ramdisk drive-based SCSI emulation, faster ramdisk,
    |              using direct memory mapping (use sagaciously).
    |
    + Storage Objects
        The actual Storage Objects that will be exported via iSCSI LUNS.
        . iblock objects need a name, and the path to a disk device.
        . fileio objects need a name, the path to a file and a size.
        . rd_dr and rd_mcp objects only need a name and a size.

The LIO object hierarchy can be populated either manually by using the HBA and storage contexts of targetcli, or by directly referencing disks or files when creating LUNs. The latter requires less commands, but is not as flexible as manually creating the underlying storage objects.

Manual storage object creation should, however, only be necessary for:

Contact

Please direct all technical discussion on targetcli to the target-devel mailing list (post, subscribe, list info, gmane archive).

If you need assistance, or if you have an SLA with Datera, Inc., please contact the company at:

For more information, please see Support.

See also

External links

Retrieved from "http://linux-iscsi.org/wiki/Targetcli"
Personal tools
Namespaces
Variants
Actions
Navigation
Toolbox
Google AdSense