Commit 2cb79266 authored by Matthew R. Ochs's avatar Matthew R. Ochs Committed by James Bottomley

cxlflash: Virtual LUN support

Add support for physical LUN segmentation (virtual LUNs) to device
driver supporting the IBM CXL Flash adapter. This patch allows user
space applications to virtually segment a physical LUN into N virtual
LUNs, taking advantage of the translation features provided by this
adapter.
Signed-off-by: default avatarMatthew R. Ochs <mrochs@linux.vnet.ibm.com>
Signed-off-by: default avatarManoj N. Kumar <manoj@linux.vnet.ibm.com>
Reviewed-by: default avatarMichael Neuling <mikey@neuling.org>
Reviewed-by: default avatarWen Xiong <wenxiong@linux.vnet.ibm.com>
Signed-off-by: default avatarJames Bottomley <JBottomley@Odin.com>
parent 65be2c79
...@@ -163,7 +163,8 @@ DK_CXLFLASH_ATTACH ...@@ -163,7 +163,8 @@ DK_CXLFLASH_ATTACH
- These tokens are only valid for the process under which they - These tokens are only valid for the process under which they
were created. The child of a forked process cannot continue were created. The child of a forked process cannot continue
to use the context id or file descriptor created by its parent. to use the context id or file descriptor created by its parent
(see DK_CXLFLASH_VLUN_CLONE for further details).
- These tokens are only valid for the lifetime of the context and - These tokens are only valid for the lifetime of the context and
the process under which they were created. Once either is the process under which they were created. Once either is
...@@ -193,6 +194,45 @@ DK_CXLFLASH_USER_DIRECT ...@@ -193,6 +194,45 @@ DK_CXLFLASH_USER_DIRECT
treated as a resource handle that is returned to the user. The user treated as a resource handle that is returned to the user. The user
is then able to use the handle to reference the LUN during I/O. is then able to use the handle to reference the LUN during I/O.
DK_CXLFLASH_USER_VIRTUAL
------------------------
This ioctl is responsible for transitioning the LUN to virtual mode
of access and configuring the AFU for virtual access from user space
on a per-context basis. Additionally, the block size and last logical
block address (LBA) are returned to the user.
As mentioned previously, when operating in user space access mode,
LUNs may be accessed in whole or in part. Only one mode is allowed
at a time and if one mode is active (outstanding references exist),
requests to use the LUN in a different mode are denied.
The AFU is configured for virtual access from user space by adding
an entry to the AFU's resource handle table. The index of the entry
is treated as a resource handle that is returned to the user. The
user is then able to use the handle to reference the LUN during I/O.
By default, the virtual LUN is created with a size of 0. The user
would need to use the DK_CXLFLASH_VLUN_RESIZE ioctl to adjust the grow
the virtual LUN to a desired size. To avoid having to perform this
resize for the initial creation of the virtual LUN, the user has the
option of specifying a size as part of the DK_CXLFLASH_USER_VIRTUAL
ioctl, such that when success is returned to the user, the
resource handle that is provided is already referencing provisioned
storage. This is reflected by the last LBA being a non-zero value.
DK_CXLFLASH_VLUN_RESIZE
-----------------------
This ioctl is responsible for resizing a previously created virtual
LUN and will fail if invoked upon a LUN that is not in virtual
mode. Upon success, an updated last LBA is returned to the user
indicating the new size of the virtual LUN associated with the
resource handle.
The partitioning of virtual LUNs is jointly mediated by the cxlflash
driver and the AFU. An allocation table is kept for each LUN that is
operating in the virtual mode and used to program a LUN translation
table that the AFU references when provided with a resource handle.
DK_CXLFLASH_RELEASE DK_CXLFLASH_RELEASE
------------------- -------------------
This ioctl is responsible for releasing a previously obtained This ioctl is responsible for releasing a previously obtained
...@@ -214,6 +254,27 @@ DK_CXLFLASH_DETACH ...@@ -214,6 +254,27 @@ DK_CXLFLASH_DETACH
success, all "tokens" which had been provided to the user from the success, all "tokens" which had been provided to the user from the
DK_CXLFLASH_ATTACH onward are no longer valid. DK_CXLFLASH_ATTACH onward are no longer valid.
DK_CXLFLASH_VLUN_CLONE
----------------------
This ioctl is responsible for cloning a previously created
context to a more recently created context. It exists solely to
support maintaining user space access to storage after a process
forks. Upon success, the child process (which invoked the ioctl)
will have access to the same LUNs via the same resource handle(s)
and fd2 as the parent, but under a different context.
Context sharing across processes is not supported with CXL and
therefore each fork must be met with establishing a new context
for the child process. This ioctl simplifies the state management
and playback required by a user in such a scenario. When a process
forks, child process can clone the parents context by first creating
a context (via DK_CXLFLASH_ATTACH) and then using this ioctl to
perform the clone from the parent to the child.
The clone itself is fairly simple. The resource handle and lun
translation tables are copied from the parent context to the child's
and then synced with the AFU.
DK_CXLFLASH_VERIFY DK_CXLFLASH_VERIFY
------------------ ------------------
This ioctl is used to detect various changes such as the capacity of This ioctl is used to detect various changes such as the capacity of
......
obj-$(CONFIG_CXLFLASH) += cxlflash.o obj-$(CONFIG_CXLFLASH) += cxlflash.o
cxlflash-y += main.o superpipe.o lunmgt.o cxlflash-y += main.o superpipe.o lunmgt.o vlun.o
...@@ -116,6 +116,9 @@ struct cxlflash_cfg { ...@@ -116,6 +116,9 @@ struct cxlflash_cfg {
atomic_t num_user_contexts; atomic_t num_user_contexts;
/* Parameters that are LUN table related */
int last_lun_index[CXLFLASH_NUM_FC_PORTS];
int promote_lun_index;
struct list_head lluns; /* list of llun_info structs */ struct list_head lluns; /* list of llun_info structs */
wait_queue_head_t tmf_waitq; wait_queue_head_t tmf_waitq;
...@@ -200,5 +203,6 @@ int cxlflash_ioctl(struct scsi_device *, int, void __user *); ...@@ -200,5 +203,6 @@ int cxlflash_ioctl(struct scsi_device *, int, void __user *);
void cxlflash_stop_term_user_contexts(struct cxlflash_cfg *); void cxlflash_stop_term_user_contexts(struct cxlflash_cfg *);
int cxlflash_mark_contexts_error(struct cxlflash_cfg *); int cxlflash_mark_contexts_error(struct cxlflash_cfg *);
void cxlflash_term_local_luns(struct cxlflash_cfg *); void cxlflash_term_local_luns(struct cxlflash_cfg *);
void cxlflash_restore_luntable(struct cxlflash_cfg *);
#endif /* ifndef _CXLFLASH_COMMON_H */ #endif /* ifndef _CXLFLASH_COMMON_H */
...@@ -20,6 +20,7 @@ ...@@ -20,6 +20,7 @@
#include "sislite.h" #include "sislite.h"
#include "common.h" #include "common.h"
#include "vlun.h"
#include "superpipe.h" #include "superpipe.h"
/** /**
...@@ -42,6 +43,7 @@ static struct llun_info *create_local(struct scsi_device *sdev, u8 *wwid) ...@@ -42,6 +43,7 @@ static struct llun_info *create_local(struct scsi_device *sdev, u8 *wwid)
lli->sdev = sdev; lli->sdev = sdev;
lli->newly_created = true; lli->newly_created = true;
lli->host_no = sdev->host->host_no; lli->host_no = sdev->host->host_no;
lli->in_table = false;
memcpy(lli->wwid, wwid, DK_CXLFLASH_MANAGE_LUN_WWID_LEN); memcpy(lli->wwid, wwid, DK_CXLFLASH_MANAGE_LUN_WWID_LEN);
out: out:
...@@ -208,6 +210,7 @@ void cxlflash_term_global_luns(void) ...@@ -208,6 +210,7 @@ void cxlflash_term_global_luns(void)
mutex_lock(&global.mutex); mutex_lock(&global.mutex);
list_for_each_entry_safe(gli, temp, &global.gluns, list) { list_for_each_entry_safe(gli, temp, &global.gluns, list) {
list_del(&gli->list); list_del(&gli->list);
cxlflash_ba_terminate(&gli->blka.ba_lun);
kfree(gli); kfree(gli);
} }
mutex_unlock(&global.mutex); mutex_unlock(&global.mutex);
......
...@@ -1989,6 +1989,8 @@ static int init_afu(struct cxlflash_cfg *cfg) ...@@ -1989,6 +1989,8 @@ static int init_afu(struct cxlflash_cfg *cfg)
afu_err_intr_init(cfg->afu); afu_err_intr_init(cfg->afu);
atomic64_set(&afu->room, readq_be(&afu->host_map->cmd_room)); atomic64_set(&afu->room, readq_be(&afu->host_map->cmd_room));
/* Restore the LUN mappings */
cxlflash_restore_luntable(cfg);
err1: err1:
pr_debug("%s: returning rc=%d\n", __func__, rc); pr_debug("%s: returning rc=%d\n", __func__, rc);
return rc; return rc;
...@@ -2286,6 +2288,17 @@ static int cxlflash_probe(struct pci_dev *pdev, ...@@ -2286,6 +2288,17 @@ static int cxlflash_probe(struct pci_dev *pdev,
cfg->init_state = INIT_STATE_NONE; cfg->init_state = INIT_STATE_NONE;
cfg->dev = pdev; cfg->dev = pdev;
/*
* The promoted LUNs move to the top of the LUN table. The rest stay
* on the bottom half. The bottom half grows from the end
* (index = 255), whereas the top half grows from the beginning
* (index = 0).
*/
cfg->promote_lun_index = 0;
cfg->last_lun_index[0] = CXLFLASH_NUM_VLUNS/2 - 1;
cfg->last_lun_index[1] = CXLFLASH_NUM_VLUNS/2 - 1;
cfg->dev_id = (struct pci_device_id *)dev_id; cfg->dev_id = (struct pci_device_id *)dev_id;
cfg->mcctx = NULL; cfg->mcctx = NULL;
......
...@@ -397,16 +397,17 @@ struct cxlflash_afu_map { ...@@ -397,16 +397,17 @@ struct cxlflash_afu_map {
}; };
}; };
/* LBA translation control blocks */ /*
* LXT - LBA Translation Table
* LXT control blocks
*/
struct sisl_lxt_entry { struct sisl_lxt_entry {
u64 rlba_base; /* bits 0:47 is base u64 rlba_base; /* bits 0:47 is base
* b48:55 is lun index * b48:55 is lun index
* b58:59 is write & read perms * b58:59 is write & read perms
* (if no perm, afu_rc=0x15) * (if no perm, afu_rc=0x15)
* b60:63 is port_sel mask * b60:63 is port_sel mask
*/ */
}; };
/* /*
...@@ -465,4 +466,7 @@ struct sisl_rht_entry_f1 { ...@@ -465,4 +466,7 @@ struct sisl_rht_entry_f1 {
#define TMF_LUN_RESET 0x1U #define TMF_LUN_RESET 0x1U
#define TMF_CLEAR_ACA 0x2U #define TMF_CLEAR_ACA 0x2U
#define SISLITE_MAX_WS_BLOCKS 512
#endif /* _SISLITE_H */ #endif /* _SISLITE_H */
...@@ -26,10 +26,24 @@ ...@@ -26,10 +26,24 @@
#include "sislite.h" #include "sislite.h"
#include "common.h" #include "common.h"
#include "vlun.h"
#include "superpipe.h" #include "superpipe.h"
struct cxlflash_global global; struct cxlflash_global global;
/**
* marshal_rele_to_resize() - translate release to resize structure
* @rele: Source structure from which to translate/copy.
* @resize: Destination structure for the translate/copy.
*/
static void marshal_rele_to_resize(struct dk_cxlflash_release *release,
struct dk_cxlflash_resize *resize)
{
resize->hdr = release->hdr;
resize->context_id = release->context_id;
resize->rsrc_handle = release->rsrc_handle;
}
/** /**
* marshal_det_to_rele() - translate detach to release structure * marshal_det_to_rele() - translate detach to release structure
* @detach: Destination structure for the translate/copy. * @detach: Destination structure for the translate/copy.
...@@ -449,6 +463,7 @@ void rhte_checkin(struct ctx_info *ctxi, ...@@ -449,6 +463,7 @@ void rhte_checkin(struct ctx_info *ctxi,
rhte->fp = 0; rhte->fp = 0;
ctxi->rht_out--; ctxi->rht_out--;
ctxi->rht_lun[rsrc_handle] = NULL; ctxi->rht_lun[rsrc_handle] = NULL;
ctxi->rht_needs_ws[rsrc_handle] = false;
} }
/** /**
...@@ -526,13 +541,21 @@ int cxlflash_lun_attach(struct glun_info *gli, enum lun_mode mode, bool locked) ...@@ -526,13 +541,21 @@ int cxlflash_lun_attach(struct glun_info *gli, enum lun_mode mode, bool locked)
/** /**
* cxlflash_lun_detach() - detaches a user from a LUN and resets the LUN's mode * cxlflash_lun_detach() - detaches a user from a LUN and resets the LUN's mode
* @gli: LUN to detach. * @gli: LUN to detach.
*
* When resetting the mode, terminate block allocation resources as they
* are no longer required (service is safe to call even when block allocation
* resources were not present - such as when transitioning from physical mode).
* These resources will be reallocated when needed (subsequent transition to
* virtual mode).
*/ */
void cxlflash_lun_detach(struct glun_info *gli) void cxlflash_lun_detach(struct glun_info *gli)
{ {
mutex_lock(&gli->mutex); mutex_lock(&gli->mutex);
WARN_ON(gli->mode == MODE_NONE); WARN_ON(gli->mode == MODE_NONE);
if (--gli->users == 0) if (--gli->users == 0) {
gli->mode = MODE_NONE; gli->mode = MODE_NONE;
cxlflash_ba_terminate(&gli->blka.ba_lun);
}
pr_debug("%s: gli->users=%u\n", __func__, gli->users); pr_debug("%s: gli->users=%u\n", __func__, gli->users);
WARN_ON(gli->users < 0); WARN_ON(gli->users < 0);
mutex_unlock(&gli->mutex); mutex_unlock(&gli->mutex);
...@@ -544,10 +567,12 @@ void cxlflash_lun_detach(struct glun_info *gli) ...@@ -544,10 +567,12 @@ void cxlflash_lun_detach(struct glun_info *gli)
* @ctxi: Context owning resources. * @ctxi: Context owning resources.
* @release: Release ioctl data structure. * @release: Release ioctl data structure.
* *
* Note that the AFU sync should _not_ be performed when the context is sitting * For LUNs in virtual mode, the virtual LUN associated with the specified
* on the error recovery list. A context on the error recovery list is not known * resource handle is resized to 0 prior to releasing the RHTE. Note that the
* to the AFU due to reset. When the context is recovered, it will be reattached * AFU sync should _not_ be performed when the context is sitting on the error
* and made known again to the AFU. * recovery list. A context on the error recovery list is not known to the AFU
* due to reset. When the context is recovered, it will be reattached and made
* known again to the AFU.
* *
* Return: 0 on success, -errno on failure * Return: 0 on success, -errno on failure
*/ */
...@@ -562,6 +587,7 @@ int _cxlflash_disk_release(struct scsi_device *sdev, ...@@ -562,6 +587,7 @@ int _cxlflash_disk_release(struct scsi_device *sdev,
struct afu *afu = cfg->afu; struct afu *afu = cfg->afu;
bool put_ctx = false; bool put_ctx = false;
struct dk_cxlflash_resize size;
res_hndl_t rhndl = release->rsrc_handle; res_hndl_t rhndl = release->rsrc_handle;
int rc = 0; int rc = 0;
...@@ -594,7 +620,24 @@ int _cxlflash_disk_release(struct scsi_device *sdev, ...@@ -594,7 +620,24 @@ int _cxlflash_disk_release(struct scsi_device *sdev,
goto out; goto out;
} }
/*
* Resize to 0 for virtual LUNS by setting the size
* to 0. This will clear LXT_START and LXT_CNT fields
* in the RHT entry and properly sync with the AFU.
*
* Afterwards we clear the remaining fields.
*/
switch (gli->mode) { switch (gli->mode) {
case MODE_VIRTUAL:
marshal_rele_to_resize(release, &size);
size.req_size = 0;
rc = _cxlflash_vlun_resize(sdev, ctxi, &size);
if (rc) {
dev_dbg(dev, "%s: resize failed rc %d\n", __func__, rc);
goto out;
}
break;
case MODE_PHYSICAL: case MODE_PHYSICAL:
/* /*
* Clear the Format 1 RHT entry for direct access * Clear the Format 1 RHT entry for direct access
...@@ -666,6 +709,7 @@ static void destroy_context(struct cxlflash_cfg *cfg, ...@@ -666,6 +709,7 @@ static void destroy_context(struct cxlflash_cfg *cfg,
/* Free memory associated with context */ /* Free memory associated with context */
free_page((ulong)ctxi->rht_start); free_page((ulong)ctxi->rht_start);
kfree(ctxi->rht_needs_ws);
kfree(ctxi->rht_lun); kfree(ctxi->rht_lun);
kfree(ctxi); kfree(ctxi);
atomic_dec_if_positive(&cfg->num_user_contexts); atomic_dec_if_positive(&cfg->num_user_contexts);
...@@ -693,11 +737,13 @@ static struct ctx_info *create_context(struct cxlflash_cfg *cfg, ...@@ -693,11 +737,13 @@ static struct ctx_info *create_context(struct cxlflash_cfg *cfg,
struct afu *afu = cfg->afu; struct afu *afu = cfg->afu;
struct ctx_info *ctxi = NULL; struct ctx_info *ctxi = NULL;
struct llun_info **lli = NULL; struct llun_info **lli = NULL;
bool *ws = NULL;
struct sisl_rht_entry *rhte; struct sisl_rht_entry *rhte;
ctxi = kzalloc(sizeof(*ctxi), GFP_KERNEL); ctxi = kzalloc(sizeof(*ctxi), GFP_KERNEL);
lli = kzalloc((MAX_RHT_PER_CONTEXT * sizeof(*lli)), GFP_KERNEL); lli = kzalloc((MAX_RHT_PER_CONTEXT * sizeof(*lli)), GFP_KERNEL);
if (unlikely(!ctxi || !lli)) { ws = kzalloc((MAX_RHT_PER_CONTEXT * sizeof(*ws)), GFP_KERNEL);
if (unlikely(!ctxi || !lli || !ws)) {
dev_err(dev, "%s: Unable to allocate context!\n", __func__); dev_err(dev, "%s: Unable to allocate context!\n", __func__);
goto err; goto err;
} }
...@@ -709,6 +755,7 @@ static struct ctx_info *create_context(struct cxlflash_cfg *cfg, ...@@ -709,6 +755,7 @@ static struct ctx_info *create_context(struct cxlflash_cfg *cfg,
} }
ctxi->rht_lun = lli; ctxi->rht_lun = lli;
ctxi->rht_needs_ws = ws;
ctxi->rht_start = rhte; ctxi->rht_start = rhte;
ctxi->rht_perms = perms; ctxi->rht_perms = perms;
...@@ -728,6 +775,7 @@ static struct ctx_info *create_context(struct cxlflash_cfg *cfg, ...@@ -728,6 +775,7 @@ static struct ctx_info *create_context(struct cxlflash_cfg *cfg,
return ctxi; return ctxi;
err: err:
kfree(ws);
kfree(lli); kfree(lli);
kfree(ctxi); kfree(ctxi);
ctxi = NULL; ctxi = NULL;
...@@ -1729,6 +1777,12 @@ static int cxlflash_disk_verify(struct scsi_device *sdev, ...@@ -1729,6 +1777,12 @@ static int cxlflash_disk_verify(struct scsi_device *sdev,
case MODE_PHYSICAL: case MODE_PHYSICAL:
last_lba = gli->max_lba; last_lba = gli->max_lba;
break; break;
case MODE_VIRTUAL:
/* Cast lxt_cnt to u64 for multiply to be treated as 64bit op */
last_lba = ((u64)rhte->lxt_cnt * MC_CHUNK_SIZE * gli->blk_len);
last_lba /= CXLFLASH_BLOCK_SIZE;
last_lba--;
break;
default: default:
WARN(1, "Unsupported LUN mode!"); WARN(1, "Unsupported LUN mode!");
} }
...@@ -1756,12 +1810,18 @@ static char *decode_ioctl(int cmd) ...@@ -1756,12 +1810,18 @@ static char *decode_ioctl(int cmd)
return __stringify_1(DK_CXLFLASH_ATTACH); return __stringify_1(DK_CXLFLASH_ATTACH);
case DK_CXLFLASH_USER_DIRECT: case DK_CXLFLASH_USER_DIRECT:
return __stringify_1(DK_CXLFLASH_USER_DIRECT); return __stringify_1(DK_CXLFLASH_USER_DIRECT);
case DK_CXLFLASH_USER_VIRTUAL:
return __stringify_1(DK_CXLFLASH_USER_VIRTUAL);
case DK_CXLFLASH_VLUN_RESIZE:
return __stringify_1(DK_CXLFLASH_VLUN_RESIZE);
case DK_CXLFLASH_RELEASE: case DK_CXLFLASH_RELEASE:
return __stringify_1(DK_CXLFLASH_RELEASE); return __stringify_1(DK_CXLFLASH_RELEASE);
case DK_CXLFLASH_DETACH: case DK_CXLFLASH_DETACH:
return __stringify_1(DK_CXLFLASH_DETACH); return __stringify_1(DK_CXLFLASH_DETACH);
case DK_CXLFLASH_VERIFY: case DK_CXLFLASH_VERIFY:
return __stringify_1(DK_CXLFLASH_VERIFY); return __stringify_1(DK_CXLFLASH_VERIFY);
case DK_CXLFLASH_VLUN_CLONE:
return __stringify_1(DK_CXLFLASH_VLUN_CLONE);
case DK_CXLFLASH_RECOVER_AFU: case DK_CXLFLASH_RECOVER_AFU:
return __stringify_1(DK_CXLFLASH_RECOVER_AFU); return __stringify_1(DK_CXLFLASH_RECOVER_AFU);
case DK_CXLFLASH_MANAGE_LUN: case DK_CXLFLASH_MANAGE_LUN:
...@@ -1876,6 +1936,7 @@ static int ioctl_common(struct scsi_device *sdev, int cmd) ...@@ -1876,6 +1936,7 @@ static int ioctl_common(struct scsi_device *sdev, int cmd)
rc = check_state(cfg); rc = check_state(cfg);
if (unlikely(rc) && (cfg->state == STATE_FAILTERM)) { if (unlikely(rc) && (cfg->state == STATE_FAILTERM)) {
switch (cmd) { switch (cmd) {
case DK_CXLFLASH_VLUN_RESIZE:
case DK_CXLFLASH_RELEASE: case DK_CXLFLASH_RELEASE:
case DK_CXLFLASH_DETACH: case DK_CXLFLASH_DETACH:
dev_dbg(dev, "%s: Command override! (%d)\n", dev_dbg(dev, "%s: Command override! (%d)\n",
...@@ -1923,12 +1984,18 @@ int cxlflash_ioctl(struct scsi_device *sdev, int cmd, void __user *arg) ...@@ -1923,12 +1984,18 @@ int cxlflash_ioctl(struct scsi_device *sdev, int cmd, void __user *arg)
{sizeof(struct dk_cxlflash_verify), (sioctl)cxlflash_disk_verify}, {sizeof(struct dk_cxlflash_verify), (sioctl)cxlflash_disk_verify},
{sizeof(struct dk_cxlflash_recover_afu), (sioctl)cxlflash_afu_recover}, {sizeof(struct dk_cxlflash_recover_afu), (sioctl)cxlflash_afu_recover},
{sizeof(struct dk_cxlflash_manage_lun), (sioctl)cxlflash_manage_lun}, {sizeof(struct dk_cxlflash_manage_lun), (sioctl)cxlflash_manage_lun},
{sizeof(struct dk_cxlflash_uvirtual), cxlflash_disk_virtual_open},
{sizeof(struct dk_cxlflash_resize), (sioctl)cxlflash_vlun_resize},
{sizeof(struct dk_cxlflash_clone), (sioctl)cxlflash_disk_clone},
}; };
/* Restrict command set to physical support only for internal LUN */ /* Restrict command set to physical support only for internal LUN */
if (afu->internal_lun) if (afu->internal_lun)
switch (cmd) { switch (cmd) {
case DK_CXLFLASH_RELEASE: case DK_CXLFLASH_RELEASE:
case DK_CXLFLASH_USER_VIRTUAL:
case DK_CXLFLASH_VLUN_RESIZE:
case DK_CXLFLASH_VLUN_CLONE:
dev_dbg(dev, "%s: %s not supported for lun_mode=%d\n", dev_dbg(dev, "%s: %s not supported for lun_mode=%d\n",
__func__, decode_ioctl(cmd), afu->internal_lun); __func__, decode_ioctl(cmd), afu->internal_lun);
rc = -EINVAL; rc = -EINVAL;
...@@ -1942,6 +2009,9 @@ int cxlflash_ioctl(struct scsi_device *sdev, int cmd, void __user *arg) ...@@ -1942,6 +2009,9 @@ int cxlflash_ioctl(struct scsi_device *sdev, int cmd, void __user *arg)
case DK_CXLFLASH_DETACH: case DK_CXLFLASH_DETACH:
case DK_CXLFLASH_VERIFY: case DK_CXLFLASH_VERIFY:
case DK_CXLFLASH_RECOVER_AFU: case DK_CXLFLASH_RECOVER_AFU:
case DK_CXLFLASH_USER_VIRTUAL:
case DK_CXLFLASH_VLUN_RESIZE:
case DK_CXLFLASH_VLUN_CLONE:
dev_dbg(dev, "%s: %s (%08X) on dev(%d/%d/%d/%llu)\n", dev_dbg(dev, "%s: %s (%08X) on dev(%d/%d/%d/%llu)\n",
__func__, decode_ioctl(cmd), cmd, shost->host_no, __func__, decode_ioctl(cmd), cmd, shost->host_no,
sdev->channel, sdev->id, sdev->lun); sdev->channel, sdev->id, sdev->lun);
......
...@@ -31,9 +31,11 @@ extern struct cxlflash_global global; ...@@ -31,9 +31,11 @@ extern struct cxlflash_global global;
#define MC_DISCOVERY_TIMEOUT 5 /* 5 secs */ #define MC_DISCOVERY_TIMEOUT 5 /* 5 secs */
#define CHAN2PORT(_x) ((_x) + 1) #define CHAN2PORT(_x) ((_x) + 1)
#define PORT2CHAN(_x) ((_x) - 1)
enum lun_mode { enum lun_mode {
MODE_NONE = 0, MODE_NONE = 0,
MODE_VIRTUAL,
MODE_PHYSICAL MODE_PHYSICAL
}; };
...@@ -41,13 +43,14 @@ enum lun_mode { ...@@ -41,13 +43,14 @@ enum lun_mode {
struct glun_info { struct glun_info {
u64 max_lba; /* from read cap(16) */ u64 max_lba; /* from read cap(16) */
u32 blk_len; /* from read cap(16) */ u32 blk_len; /* from read cap(16) */
enum lun_mode mode; /* NONE, PHYSICAL */ enum lun_mode mode; /* NONE, VIRTUAL, PHYSICAL */
int users; /* Number of users w/ references to LUN */ int users; /* Number of users w/ references to LUN */
u8 wwid[16]; u8 wwid[16];
struct mutex mutex; struct mutex mutex;
struct blka blka;
struct list_head list; struct list_head list;
}; };
...@@ -58,6 +61,7 @@ struct llun_info { ...@@ -58,6 +61,7 @@ struct llun_info {
u32 host_no; /* host_no from Scsi_host */ u32 host_no; /* host_no from Scsi_host */
u32 port_sel; /* What port to use for this LUN */ u32 port_sel; /* What port to use for this LUN */
bool newly_created; /* Whether the LUN was just discovered */ bool newly_created; /* Whether the LUN was just discovered */
bool in_table; /* Whether a LUN table entry was created */
u8 wwid[16]; /* Keep a duplicate copy here? */ u8 wwid[16]; /* Keep a duplicate copy here? */
...@@ -90,6 +94,7 @@ struct ctx_info { ...@@ -90,6 +94,7 @@ struct ctx_info {
u32 rht_out; /* Number of checked out RHT entries */ u32 rht_out; /* Number of checked out RHT entries */
u32 rht_perms; /* User-defined permissions for RHT entries */ u32 rht_perms; /* User-defined permissions for RHT entries */
struct llun_info **rht_lun; /* Mapping of RHT entries to LUNs */ struct llun_info **rht_lun; /* Mapping of RHT entries to LUNs */
bool *rht_needs_ws; /* User-desired write-same function per RHTE */
struct cxl_ioctl_start_work work; struct cxl_ioctl_start_work work;
u64 ctxid; u64 ctxid;
...@@ -111,10 +116,18 @@ struct cxlflash_global { ...@@ -111,10 +116,18 @@ struct cxlflash_global {
struct page *err_page; /* One page of all 0xF for error notification */ struct page *err_page; /* One page of all 0xF for error notification */
}; };
int cxlflash_vlun_resize(struct scsi_device *, struct dk_cxlflash_resize *);
int _cxlflash_vlun_resize(struct scsi_device *, struct ctx_info *,
struct dk_cxlflash_resize *);
int cxlflash_disk_release(struct scsi_device *, struct dk_cxlflash_release *); int cxlflash_disk_release(struct scsi_device *, struct dk_cxlflash_release *);
int _cxlflash_disk_release(struct scsi_device *, struct ctx_info *, int _cxlflash_disk_release(struct scsi_device *, struct ctx_info *,
struct dk_cxlflash_release *); struct dk_cxlflash_release *);
int cxlflash_disk_clone(struct scsi_device *, struct dk_cxlflash_clone *);
int cxlflash_disk_virtual_open(struct scsi_device *, void *);
int cxlflash_lun_attach(struct glun_info *, enum lun_mode, bool); int cxlflash_lun_attach(struct glun_info *, enum lun_mode, bool);
void cxlflash_lun_detach(struct glun_info *); void cxlflash_lun_detach(struct glun_info *);
...@@ -127,6 +140,8 @@ struct sisl_rht_entry *get_rhte(struct ctx_info *, res_hndl_t, ...@@ -127,6 +140,8 @@ struct sisl_rht_entry *get_rhte(struct ctx_info *, res_hndl_t,
struct sisl_rht_entry *rhte_checkout(struct ctx_info *, struct llun_info *); struct sisl_rht_entry *rhte_checkout(struct ctx_info *, struct llun_info *);
void rhte_checkin(struct ctx_info *, struct sisl_rht_entry *); void rhte_checkin(struct ctx_info *, struct sisl_rht_entry *);
void cxlflash_ba_terminate(struct ba_lun *);
int cxlflash_manage_lun(struct scsi_device *, struct dk_cxlflash_manage_lun *); int cxlflash_manage_lun(struct scsi_device *, struct dk_cxlflash_manage_lun *);
#endif /* ifndef _CXLFLASH_SUPERPIPE_H */ #endif /* ifndef _CXLFLASH_SUPERPIPE_H */
This diff is collapsed.
/*
* CXL Flash Device Driver
*
* Written by: Manoj N. Kumar <manoj@linux.vnet.ibm.com>, IBM Corporation
* Matthew R. Ochs <mrochs@linux.vnet.ibm.com>, IBM Corporation
*
* Copyright (C) 2015 IBM Corporation
*
* This program is free software; you can redistribute it and/or
* modify it under the terms of the GNU General Public License
* as published by the Free Software Foundation; either version
* 2 of the License, or (at your option) any later version.
*/
#ifndef _CXLFLASH_VLUN_H
#define _CXLFLASH_VLUN_H
/* RHT - Resource Handle Table */
#define MC_RHT_NMASK 16 /* in bits */
#define MC_CHUNK_SHIFT MC_RHT_NMASK /* shift to go from LBA to chunk# */
#define HIBIT (BITS_PER_LONG - 1)
#define MAX_AUN_CLONE_CNT 0xFF
/*
* LXT - LBA Translation Table
*
* +-------+-------+-------+-------+-------+-------+-------+---+---+
* | RLBA_BASE |LUN_IDX| P |SEL|
* +-------+-------+-------+-------+-------+-------+-------+---+---+
*
* The LXT Entry contains the physical LBA where the chunk starts (RLBA_BASE).
* AFU ORes the low order bits from the virtual LBA (offset into the chunk)
* with RLBA_BASE. The result is the physical LBA to be sent to storage.
* The LXT Entry also contains an index to a LUN TBL and a bitmask of which
* outgoing (FC) * ports can be selected. The port select bit-mask is ANDed
* with a global port select bit-mask maintained by the driver.
* In addition, it has permission bits that are ANDed with the
* RHT permissions to arrive at the final permissions for the chunk.
*
* LXT tables are allocated dynamically in groups. This is done to avoid
* a malloc/free overhead each time the LXT has to grow or shrink.
*
* Based on the current lxt_cnt (used), it is always possible to know
* how many are allocated (used+free). The number of allocated entries is
* not stored anywhere.
*
* The LXT table is re-allocated whenever it needs to cross into another group.
*/
#define LXT_GROUP_SIZE 8
#define LXT_NUM_GROUPS(lxt_cnt) (((lxt_cnt) + 7)/8) /* alloc'ed groups */
#define LXT_LUNIDX_SHIFT 8 /* LXT entry, shift for LUN index */
#define LXT_PERM_SHIFT 4 /* LXT entry, shift for permission bits */
struct ba_lun_info {
u64 *lun_alloc_map;
u32 lun_bmap_size;
u32 total_aus;
u64 free_aun_cnt;
/* indices to be used for elevator lookup of free map */
u32 free_low_idx;
u32 free_curr_idx;
u32 free_high_idx;
u8 *aun_clone_map;
};
struct ba_lun {
u64 lun_id;
u64 wwpn;
size_t lsize; /* LUN size in number of LBAs */
size_t lba_size; /* LBA size in number of bytes */
size_t au_size; /* Allocation Unit size in number of LBAs */
struct ba_lun_info *ba_lun_handle;
};
/* Block Allocator */
struct blka {
struct ba_lun ba_lun;
u64 nchunk; /* number of chunks */
struct mutex mutex;
};
#endif /* ifndef _CXLFLASH_SUPERPIPE_H */
...@@ -71,6 +71,17 @@ struct dk_cxlflash_udirect { ...@@ -71,6 +71,17 @@ struct dk_cxlflash_udirect {
__u64 reserved[8]; /* Reserved for future use */ __u64 reserved[8]; /* Reserved for future use */
}; };
#define DK_CXLFLASH_UVIRTUAL_NEED_WRITE_SAME 0x8000000000000000ULL
struct dk_cxlflash_uvirtual {
struct dk_cxlflash_hdr hdr; /* Common fields */
__u64 context_id; /* Context to own virtual resources */
__u64 lun_size; /* Requested size, in 4K blocks */
__u64 rsrc_handle; /* Returned resource handle */
__u64 last_lba; /* Returned last LBA of LUN */
__u64 reserved[8]; /* Reserved for future use */
};
struct dk_cxlflash_release { struct dk_cxlflash_release {
struct dk_cxlflash_hdr hdr; /* Common fields */ struct dk_cxlflash_hdr hdr; /* Common fields */
__u64 context_id; /* Context owning resources */ __u64 context_id; /* Context owning resources */
...@@ -78,6 +89,23 @@ struct dk_cxlflash_release { ...@@ -78,6 +89,23 @@ struct dk_cxlflash_release {
__u64 reserved[8]; /* Reserved for future use */ __u64 reserved[8]; /* Reserved for future use */
}; };
struct dk_cxlflash_resize {
struct dk_cxlflash_hdr hdr; /* Common fields */
__u64 context_id; /* Context owning resources */
__u64 rsrc_handle; /* Resource handle of LUN to resize */
__u64 req_size; /* New requested size, in 4K blocks */
__u64 last_lba; /* Returned last LBA of LUN */
__u64 reserved[8]; /* Reserved for future use */
};
struct dk_cxlflash_clone {
struct dk_cxlflash_hdr hdr; /* Common fields */
__u64 context_id_src; /* Context to clone from */
__u64 context_id_dst; /* Context to clone to */
__u64 adap_fd_src; /* Source context adapter fd */
__u64 reserved[8]; /* Reserved for future use */
};
#define DK_CXLFLASH_VERIFY_SENSE_LEN 18 #define DK_CXLFLASH_VERIFY_SENSE_LEN 18
#define DK_CXLFLASH_VERIFY_HINT_SENSE 0x8000000000000000ULL #define DK_CXLFLASH_VERIFY_HINT_SENSE 0x8000000000000000ULL
...@@ -118,7 +146,10 @@ union cxlflash_ioctls { ...@@ -118,7 +146,10 @@ union cxlflash_ioctls {
struct dk_cxlflash_attach attach; struct dk_cxlflash_attach attach;
struct dk_cxlflash_detach detach; struct dk_cxlflash_detach detach;
struct dk_cxlflash_udirect udirect; struct dk_cxlflash_udirect udirect;
struct dk_cxlflash_uvirtual uvirtual;
struct dk_cxlflash_release release; struct dk_cxlflash_release release;
struct dk_cxlflash_resize resize;
struct dk_cxlflash_clone clone;
struct dk_cxlflash_verify verify; struct dk_cxlflash_verify verify;
struct dk_cxlflash_recover_afu recover_afu; struct dk_cxlflash_recover_afu recover_afu;
struct dk_cxlflash_manage_lun manage_lun; struct dk_cxlflash_manage_lun manage_lun;
...@@ -136,5 +167,8 @@ union cxlflash_ioctls { ...@@ -136,5 +167,8 @@ union cxlflash_ioctls {
#define DK_CXLFLASH_VERIFY CXL_IOWR(0x84, dk_cxlflash_verify) #define DK_CXLFLASH_VERIFY CXL_IOWR(0x84, dk_cxlflash_verify)
#define DK_CXLFLASH_RECOVER_AFU CXL_IOWR(0x85, dk_cxlflash_recover_afu) #define DK_CXLFLASH_RECOVER_AFU CXL_IOWR(0x85, dk_cxlflash_recover_afu)
#define DK_CXLFLASH_MANAGE_LUN CXL_IOWR(0x86, dk_cxlflash_manage_lun) #define DK_CXLFLASH_MANAGE_LUN CXL_IOWR(0x86, dk_cxlflash_manage_lun)
#define DK_CXLFLASH_USER_VIRTUAL CXL_IOWR(0x87, dk_cxlflash_uvirtual)
#define DK_CXLFLASH_VLUN_RESIZE CXL_IOWR(0x88, dk_cxlflash_resize)
#define DK_CXLFLASH_VLUN_CLONE CXL_IOWR(0x89, dk_cxlflash_clone)
#endif /* ifndef _CXLFLASH_IOCTL_H */ #endif /* ifndef _CXLFLASH_IOCTL_H */
Markdown is supported
0%
or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment