Persistent Reservations

From Linux-iSCSI
Jump to: navigation, search

SCSI-3 and SCSI-4 persistent reservations are used for a number of cluster storage configurations for I/O fencing and failover/retakeover cases using iSCSI and other SCSI fabrics like Fibre Channel and Serial Attached SCSI.

Our Linux-iSCSI Target Mode SPC-3 defined Persistent Reservations is supported in a SCSI fabric module independent method in target_core_mod in the lio-core-2.6.git source tree, you can have a look at the latest revision of the source code at drivers/target/target_core_pr.c using kernel.org git.

As of August 10th 2009 with commit and commit on v2.6.31-rc5, the TCM PR code in lio-core-2.6.git/drivers/target/target_core_pr.c is now %100 persistent reservations feature complete for SPC-4!

Also, using Asymmetric Logical Unit Assignment ALUA with Persistent Reservation in TCM/LIO v3.1 is fully supported!

Contents

What Cluster Clients that use Linux-iSCSI target mode PR have been tested..?

  • ) RHEL/CentOS v5u3 using SCSI Fencing

RHEL/CentOS uses sg3_util's sg_persist CLI operation for Write Exclusive, Registrants Only reservations and PREEMPT_AND_ABORT takeover using ext3 mounts. Testing with GFS (eg: with multiple writers) is underway.

See RHEL/Linux_Persistent_Reservations for more information about Linux/iSCSI Initiators using LIO-Target v3.0 Mode

All target_core_mod/LIO-Target v3.0 capable storage objects from IBLOCK, FILEIO, and RAMDISK_MCP/DR pass fence_scsi_test.

SPC-3 PR emulation is disabled on top of the passthrough SCSI subsystem plugin (this is a WIP for existing SCSI devices)

  • ) MSFT Cluster 2008 (uses Write Exclusive, Registrants Only and PREEMPT) using the MSFT domain validation suite.

See MSFT/Persistent_Reservations for more information

Testing LIO-Target v3.0 against other SPC-3 PR compliant systems

  • ) If you have a iSCSI Initiator capable platform that implements SPC-3 Peristent Reservation logic and would like tosee your code running against target_core_mod and LIO-Target v3.0, please let us know on the list: LIO-Target devel

What has been implemented..?

  • ) All SPC-4 defined features:

These two spc4 defined PR features been now been committed in lio-core-2.6.git running v2.6.31-rc5!

APTPL=1 (Activate Persist across Target Power Loss) is using key=value /var/target/pr/aptpl_$T10_UNIT_SERIAL using sg_persist See link for more information

SP_I_PT=1 (Specify Initiator Ports Capable) bit is supported and tested using sg_persist See link

  • ) All PROUT Service Actions:

REGISTER, RESERVE, RELEASE, CLEAR, REGISTER_AND_IGNORE, PREEMPT, PREEMPT_AND_ABORT and REGISTER_AND_MOVE

  • ) All PROUT Reservation Types:

Write Exclusive, Exclusive Access, Write Exclusive Registrants Only, Exclusive Access Registrants Only Write Exclusive All Registrants, Exclusive Access All Registrants

  • ) All PRIN Service Actions:

READ_KEYS, READ_RESERVATION, REPORT_CAPABILITIES, READ_FULL_STATUS

Whats left to enable..?

Compatible reservation handling for SPC-2 needs to be enabled for legacy environments.

ConfigFS layout for target_core_mod

  • ) This is a output of an IBLOCK exported Linux/LVM DM block_device (lvm_test0) via 
  tree /sys/kernel/config/target/core
  |-- iblock_0
  |   |-- hba_info
  |   `-- lvm_test0
  |       |-- alua_lu_gp
  |       |-- attrib
  |       |   |-- block_size
  |       |   |-- emulate_tas
  |       |   |-- emulate_ua_intlck_ctrl
  |       |   |-- hw_block_size
  |       |   |-- hw_max_sectors
  |       |   |-- hw_queue_depth
  |       |   |-- max_sectors
  |       |   |-- queue_depth
  |       |   `-- task_timeout
  |       |-- control
  |       |-- enable
  |       |-- fd
  |       |-- info
  |       |-- pr
  |       |   |-- res_holder
  |       |   |-- res_pr_all_tgt_pts
  |       |   |-- res_pr_generation
  |       |   |-- res_pr_holder_tg_port
  |       |   |-- res_pr_registered_i_pts
  |       |   |-- res_pr_type
  |       |   `-- res_type
  |       `-- wwn
  |           |-- vpd_assoc_logical_unit
  |           |-- vpd_assoc_scsi_target_device
  |           |-- vpd_assoc_target_port
  |           |-- vpd_protocol_identifier
  |           `-- vpd_unit_serial
  • ) Output of PRIN REPORT_CAPABILITES to across LIO-Target: 
  initiator# sg_persist --in --report-capabilities -v /dev/sde
      inquiry cdb: 12 00 00 00 24 00 
    LIO-ORG  IBLOCK  3.0
    Peripheral device type: disk
      Persistent Reservation In cmd: 5e 02 00 00 00 00 00 20 00 00 
  Report capabilities response:
    Compatible Reservation Handling(CRH): 0
    Specify Initiator Ports Capable(SIP_C): 0
    All Target Ports Capable(ATP_C): 1
    Persist Through Power Loss Capable(PTPL_C): 0
    Type Mask Valid(TMV): 1
    Allow commands: 1
    Persist Through Power Loss Active(PTPL_A): 0
      Support indicated in Type mask:
        Write Exclusive, all registrants: 1
        Exclusive Access, registrants only: 1
        Write Exclusive, registrants only: 1
        Exclusive Access: 1
        Write Exclusive: 1
        Exclusive Access, all registrants: 1
  • ) The iblock_0/lvm_test0/pr/ ConfigFS attributes are used to see the current reservation/registration status

for different SCSI (including LIO-Target v3.0) I_T nexuses: 

  cat /sys/kernel/config/target/core/iblock_0/lvm_test0/pr/res_*
  SPC-3 Reservation: iSCSI Initiator: iqn.1993-08.org.debian:01:2dadf92d0ef
  SPC-3 Reservation: All Target Ports registration 0x00000002
  SPC-3 Reservation: iSCSI Target Node Endpoint: iqn.2003-01.org.linux-iscsi.target.i686:sn.cff3eedbd2fd
  SPC-3 Reservation: Relative Port Identifer Tag: 2 iSCSI Portal Group Tag: 1 iSCSI Logical Unit: 0
  SPC-3 PR Registrations:
     iSCSI Node: iqn.1993-08.org.debian:01:2dadf92d0ef Key: 0x000000001234abcd PRgen: 0x00000000
     iSCSI Node: iqn.1993-08.org.debian:01:2dadf92d0ef Key: 0x000000004567ffff PRgen: 0x00000001
  SPC-3 Reservation Type: Write Exclusive Access, Registrants Only
  • ) The iblock_0/lvm_test0/wwn ConfigFS attributes are used to see world wide unique LUN information, be it

real from the SCSI device, or emulated using a admin present value into iblock_0/lvm_test0/wwn/evp_unit_serial. This is required in order to use the NAA VPD device identifers. 

  cat /sys/kernel/config/target/core/iblock_0/lvm_test0/wwn/*   
  T10 VPD Identifier Association: addressed logical unit
  T10 VPD Identifier Type: NAA
  T10 VPD Binary Device Identifier: 36001405a97e4ce21c0711de829b000c2
  T10 VPD Identifier Association: addressed logical unit
  T10 VPD Identifier Type: T10 Vendor ID based
  T10 VPD ASCII Device Identifier: LIO-ORG
  T10 VPD Unit Serial Number: a97e4ce21c0711de829b000c2943d57b
Retrieved from "http://linux-iscsi.org/index.php/Persistent_Reservations"
Personal tools