2.6.24 kernel
Bart Van Assche
The Linux-iSCSI target (kernel module) does not compile on the 2.6.24
kernel because of changes in the scatterlist API. Has a target date
for a version that compiles with the 2.6.24 kernel headers already
been set ?
Bart Van Assche.
Nicholas A. Bellinger
I will doing a 2.6.24 build for LIO on PS3-Linux this weekend now that
ps3rom.c is reporting the proper struct scsi_host_template->max_sectors.
Also in the queue are releasing updated builds for CentOS v5.1 x86_64.
I have recently upgraded one pair of core nodes on the Linux-iSCSI.org
fabric to the newest CentOS release, and minus a few minor issues with
the upgrade process, everything is looking very stable. The few issues
that I had (an lvremove issue on v4.5, and qemu-dm performance with HVM
on v5.0) have been resolved and the fabric is running much smoother for
the VMs that are providing OCFS2 cluster storage to the LIO debian and
ubuntu repositories.
I will let you know when I get 2.6.24 building. Until then, feel free
to post the build failure log here.
Many thanks for your most valuable of time,
--nab
Nicholas A. Bellinger
Ok, I have 2.6.24 running on ppc64 doing iSCSI/HD on PS3-Linux. The
changes for struct scatterlist->page moving to struct
scatterlist->page_link where pretty straightforward, considering the LIO
storage engine does not depend on struct scatterlist. I am going to
take another look at the diffs later today and make sure everything
looks corerct, and will make the commits then.
If you could test these on your setup with 2.6.24 in the non IPoIB case
(that from my previous emails I am guessing will be fine on your setup)
I would really appreciate it. I will put 2.6.24 in VM on the
Linux-iSCSI.org fabric in the upcoming days, and do some additional
testing. Getting the CentOS v5u1 x86_64 builds release are a bit higher
priority than 2.6.24, but I think that the 2.6.24 changes are reasonable
and do not cause concern with the LIO v2.9 codebase.
Have you made any futher progress on debugging the issue LIO Target with
IPoIB..?
Many thanks for your most valuable of time,
--nab
Nicholas A. Bellinger
Here are the changes to transport_memcpy_[write,read]_[contig,sg]()
respectively. This functions are legacy within v2.9 LIO SE, and are
currently unused in kernel mode because the SE core does not rely on
struct scatterlist.
Index: target/iscsi_target_transport.c
===================================================================
--- target/iscsi_target_transport.c (revision 205)
+++ target/iscsi_target_transport.c (revision 206)
@@ -4181,7 +4181,7 @@
if (length > total_length)
length = total_length;
- src = GET_ADDR_SG(sg_s, i);
+ src = GET_ADDR_SG(&sg_s[i]);
memcpy(dst, src, length);
@@ -4211,12 +4211,12 @@
if (length > total_length)
length = total_length;
- dst = GET_ADDR_SG(sg_d, i) + dst_offset;
+ dst = GET_ADDR_SG(&sg_d[i]) + dst_offset;
if (!dst)
BUG();
i++;
- src = GET_ADDR_SG(sg_s, j) + src_offset;
+ src = GET_ADDR_SG(&sg_s[j]) + src_offset;
if (!src)
BUG();
@@ -4228,7 +4228,7 @@
if (length > total_length)
length = total_length;
- dst = GET_ADDR_SG(sg_d, i) + dst_offset;
+ dst = GET_ADDR_SG(&sg_d[i]) + dst_offset;
if (!dst)
BUG();
@@ -4238,7 +4238,7 @@
} else
dst_offset = length;
- src = GET_ADDR_SG(sg_s, j) + src_offset;
+ src = GET_ADDR_SG(&sg_s[j]) + src_offset;
if (!src)
BUG();
j++;
@@ -4269,7 +4269,7 @@
if (length > total_length)
length = total_length;
- dst = GET_ADDR_SG(sg_d, i);
+ dst = GET_ADDR_SG(&sg_d[i]);
memcpy(dst, src, length);
These changes are followed up by transport_map_sg_to_mem()
and transport_map_mem_to_sg().. The latter is the default path for
LIO SE v2.9 to v2.6 Linux storage subsystems along with the other completely
virtual subsystem drivers. Note the former does reverse contigious
scatterlist array mapping to SE linked list memory which is then handed to
the SCSI transport, in the LIO case, traditional iSCSI. This is currently
unused, but may be useful in some future cases.
@@ -4299,12 +4299,12 @@
if (length > total_length)
length = total_length;
- src = GET_ADDR_SG(sg_s, i) + src_offset;
+ src = GET_ADDR_SG(&sg_s[i]) + src_offset;
if (!src)
BUG();
i++;
- dst = GET_ADDR_SG(sg_d, j) + dst_offset;
+ dst = GET_ADDR_SG(&sg_d[j]) + dst_offset;
if (!dst)
BUG();
@@ -4316,7 +4316,7 @@
if (length > total_length)
length = total_length;
- src = GET_ADDR_SG(sg_s, i) + src_offset;
+ src = GET_ADDR_SG(&sg_s[i]) + src_offset;
if (!src)
BUG();
@@ -4326,7 +4326,7 @@
} else
src_offset = length;
- dst = GET_ADDR_SG(sg_d, j) + dst_offset;
+ dst = GET_ADDR_SG(&sg_d[j]) + dst_offset;
if (!dst)
BUG();
j++;
@@ -5191,7 +5191,7 @@
INIT_LIST_HEAD(&se_mem->se_list);
if (*task_offset == 0) {
- se_mem->se_page = sg_s[j].page;
+ se_mem->se_page = GET_PAGE_SG(&sg_s[j]);
se_mem->se_off = sg_s[j].offset;
if (task_size >= sg_s[j].length)
@@ -5208,7 +5208,7 @@
if (saved_task_offset)
*task_offset = saved_task_offset;
} else {
- se_mem->se_page = sg_s[j].page;
+ se_mem->se_page = GET_PAGE_SG(&sg_s[j]);
se_mem->se_off = (*task_offset + sg_s[j].offset);
if ((sg_s[j].length - *task_offset) > task_size) {
@@ -5348,7 +5348,7 @@
while (task_size) {
if (*task_offset == 0) {
- sg[sg_no].page = se_mem->se_page;
+ SET_PAGE_SG(&sg[sg_no], se_mem->se_page);
sg[sg_no].offset = se_mem->se_off;
if (task_size >= se_mem->se_len) {
@@ -5375,7 +5375,7 @@
if (saved_task_offset)
*task_offset = saved_task_offset;
} else {
- sg[sg_no].page = se_mem->se_page;
+ SET_PAGE_SG(&sg[sg_no], se_mem->se_page);
sg[sg_no].offset = (*task_offset + se_mem->se_off);
if ((se_mem->se_len - *task_offset) > task_size) {
The SE algoritims can also potentially accept preallocated memory for
RDMA operations and then map said memory to contiguous struct
scatterlist buffers that are then mapped to the SCSI, BIO, and
[read,write]v() for struct file.
Here are the macro changes for new and legacy operation:
Index: include/iscsi_linux_defs.h
===================================================================
--- include/iscsi_linux_defs.h (revision 207)
+++ include/iscsi_linux_defs.h (revision 208)
@@ -44,15 +44,29 @@
#define inline
#endif
-/* added to address 64bit addressing issues. */
+/*
+ * 2.6.24 provides an updated struct scatterlist API. Use macros for
the new
+ * code, and use inline functions for legacy operation.
+ */
#if LINUX_VERSION_CODE >= KERNEL_VERSION(2,6,24)
# define GET_ADDR_SG(sg) sg_virt(sg)
# define GET_PAGE_SG(sg) sg_page(sg)
# define SET_PAGE_SG(sg, page) sg_assign_page(sg, page)
#else
-# define GET_ADDR_SG(sg) page_address(sg->page) + sg->offset
-# define GET_PAGE_SG(sg) sg->page
-# define SET_PAGE_SG(sg, page) sg->page = page
+#include <linux/scatterlist.h>
+static inline void *GET_ADDR_SG(struct scatterlist *sg)
+{
+ return(page_address(sg->page) + sg->offset);
+}
+static inline struct page *GET_PAGE_SG(struct scatterlist *sg)
+{
+ return(sg->page);
+}
+static inline void SET_PAGE_SG(struct scatterlist *sg, struct page
*page)
+{
+ sg->page = page;
+ return;
+}
#endif
/*
--nab
>
> >
xavia
yesterday. I tried to compile linux-iscsi and just got a bunch of
warnings and errors. I am using linux-iscsi rev 207 from today which
appears to have all the above patches already installed. I am stumpted
as to why this isn't working. Any hints or suggestions as to what is
going on or what I did wrong?
Matt
-------
Linux blackbeauty 2.6.24-cell-custom #1 SMP Tue Feb 12 01:41:20 EST
2008 ppc64 GNU/Linux
matt@blackbeauty:~/linux-iscsi/target$ make
make -C target
make[1]: Entering directory `/home/matt/linux-iscsi/target/target'
make -C /lib/modules/2.6.24-cell-custom/build SUBDIRS=/home/matt/linux-
iscsi/target/target modules CWD=/home/matt/linux-iscsi/target/target
ARCH=ppc KBUILD_VERBOSE=0
make[2]: Entering directory `/home/matt/linux-cell-20080212-2.6.24'
CC [M] /home/matt/linux-iscsi/target/target/../common/iscsi_crc.o
In file included from include/asm/mmu.h:7,
from include/asm/lppaca.h:32,
from include/asm/paca.h:20,
from include/asm/hw_irq.h:17,
from include/asm/system.h:9,
from include/linux/list.h:9,
from include/linux/preempt.h:11,
from include/linux/spinlock.h:49,
from include/linux/mmzone.h:7,
from include/linux/gfp.h:4,
from include/linux/slab.h:14,
from /home/matt/linux-iscsi/target/target/../common/
iscsi_crc.c:29:
include/asm/mmu-hash64.h: In function âhpte_encode_vâ:
include/asm/mmu-hash64.h:212: warning: left shift count >= width of
type
include/asm/mmu-hash64.h: In function âhpte_encode_râ:
include/asm/mmu-hash64.h:227: warning: integer constant is too large
for âunsigned longâ type
include/asm/mmu-hash64.h: In function âhpt_vaâ:
include/asm/mmu-hash64.h:244: warning: left shift count >= width of
type
include/asm/mmu-hash64.h:244: warning: integer constant is too large
for âunsigned longâ type
include/asm/mmu-hash64.h: In function âhpt_hashâ:
include/asm/mmu-hash64.h:259: warning: right shift count >= width of
type
include/asm/mmu-hash64.h:260: warning: integer constant is too large
for âunsigned longâ type
include/asm/mmu-hash64.h:262: warning: integer constant is too large
for âunsigned longâ type
include/asm/mmu-hash64.h: In function âget_kernel_vsidâ:
include/asm/mmu-hash64.h:437: error: âSID_SHIFTâ undeclared (first use
in this function)
include/asm/mmu-hash64.h:437: error: (Each undeclared identifier is
reported only once
include/asm/mmu-hash64.h:437: error: for each function it appears in.)
include/asm/mmu-hash64.h:437: warning: right shift count >= width of
type
include/asm/mmu-hash64.h:437: warning: left shift count >= width of
type
include/asm/mmu-hash64.h:437: warning: right shift count >= width of
type
include/asm/mmu-hash64.h:437: warning: left shift count >= width of
type
include/asm/mmu-hash64.h:438: error: âSID_SHIFT_1Tâ undeclared (first
use in this function)
include/asm/mmu-hash64.h: In function âuser_segment_sizeâ:
include/asm/mmu-hash64.h:445: error: âSID_SHIFT_1Tâ undeclared (first
use in this function)
include/asm/mmu-hash64.h: In function âget_vsidâ:
include/asm/mmu-hash64.h:455: error: âSID_SHIFTâ undeclared (first use
in this function)
include/asm/mmu-hash64.h:455: warning: right shift count >= width of
type
include/asm/mmu-hash64.h:455: warning: left shift count >= width of
type
include/asm/mmu-hash64.h:455: warning: right shift count >= width of
type
include/asm/mmu-hash64.h:455: warning: left shift count >= width of
type
include/asm/mmu-hash64.h:457: error: âSID_SHIFT_1Tâ undeclared (first
use in this function)
In file included from include/asm/spinlock.h:25,
from include/linux/spinlock.h:87,
from include/linux/mmzone.h:7,
from include/linux/gfp.h:4,
from include/linux/slab.h:14,
from /home/matt/linux-iscsi/target/target/../common/
iscsi_crc.c:29:
include/asm/iseries/hv_call.h: In function âHvCall_setVirtualDecrâ:
include/asm/iseries/hv_call.h:85: warning: integer constant is too
large for âunsigned longâ type
include/asm/iseries/hv_call.h: In function âHvCall_yieldProcessorâ:
include/asm/iseries/hv_call.h:90: warning: integer constant is too
large for âunsigned longâ type
include/asm/iseries/hv_call.h: In function
âHvCall_setEnabledInterruptsâ:
include/asm/iseries/hv_call.h:95: warning: integer constant is too
large for âunsigned longâ type
include/asm/iseries/hv_call.h: In function
âHvCall_setLogBufferFormatAndCodepageâ:
include/asm/iseries/hv_call.h:101: warning: integer constant is too
large for âunsigned longâ type
include/asm/iseries/hv_call.h: In function âHvCall_sendIPIâ:
include/asm/iseries/hv_call.h:108: warning: integer constant is too
large for âunsigned longâ type
In file included from include/linux/security.h:25,
from include/linux/mm.h:15,
from /home/matt/linux-iscsi/target/target/../include/
iscsi_linux_defs.h:27,
from /home/matt/linux-iscsi/target/target/../include/
iscsi_crc.h:21,
from /home/matt/linux-iscsi/target/target/../common/
iscsi_crc.c:35:
include/linux/fs.h: At top level:
include/linux/fs.h:909: warning: âstruct flock64â declared inside
parameter list
include/linux/fs.h:909: warning: its scope is only this definition or
declaration, which is probably not what you want
include/linux/fs.h:911: warning: âstruct flock64â declared inside
parameter list
In file included from include/asm/cputime.h:26,
from include/linux/sched.h:66,
from include/linux/security.h:32,
from include/linux/mm.h:15,
from /home/matt/linux-iscsi/target/target/../include/
iscsi_linux_defs.h:27,
from /home/matt/linux-iscsi/target/target/../include/
iscsi_crc.h:21,
from /home/matt/linux-iscsi/target/target/../common/
iscsi_crc.c:35:
include/asm/time.h: In function âget_tbâ:
include/asm/time.h:135: error: implicit declaration of function âmftbâ
In file included from include/asm/pgtable.h:13,
from include/linux/mm.h:38,
from /home/matt/linux-iscsi/target/target/../include/
iscsi_linux_defs.h:27,
from /home/matt/linux-iscsi/target/target/../include/
iscsi_crc.h:21,
from /home/matt/linux-iscsi/target/target/../common/
iscsi_crc.c:35:
include/asm/pgtable-ppc64.h:32:50: warning: "SID_SHIFT" is not defined
include/asm/pgtable-ppc64.h:33:2: error: #error TASK_SIZE_USER64
exceeds user VSID range
In file included from include/asm/pgtable.h:13,
from include/linux/mm.h:38,
from /home/matt/linux-iscsi/target/target/../include/
iscsi_linux_defs.h:27,
from /home/matt/linux-iscsi/target/target/../include/
iscsi_crc.h:21,
from /home/matt/linux-iscsi/target/target/../common/
iscsi_crc.c:35:
include/asm/pgtable-ppc64.h: In function âfind_linux_pteâ:
include/asm/pgtable-ppc64.h:429: warning: right shift count >= width
of type
In file included from include/asm/pgtable.h:40,
from include/linux/mm.h:38,
from /home/matt/linux-iscsi/target/target/../include/
iscsi_linux_defs.h:27,
from /home/matt/linux-iscsi/target/target/../include/
iscsi_crc.h:21,
from /home/matt/linux-iscsi/target/target/../common/
iscsi_crc.c:35:
include/asm-generic/pgtable.h: In function âpud_none_or_clear_badâ:
include/asm-generic/pgtable.h:181: warning: integer constant is too
large for âunsigned longâ type
include/asm-generic/pgtable.h:181: warning: comparison is always false
due to limited range of data type
include/asm-generic/pgtable.h: In function âpmd_none_or_clear_badâ:
include/asm-generic/pgtable.h:192: warning: integer constant is too
large for âunsigned longâ type
include/asm-generic/pgtable.h:192: warning: comparison is always false
due to limited range of data type
In file included from /home/matt/linux-iscsi/target/target/../include/
iscsi_linux_defs.h:27,
from /home/matt/linux-iscsi/target/target/../include/
iscsi_crc.h:21,
from /home/matt/linux-iscsi/target/target/../common/
iscsi_crc.c:35:
include/linux/mm.h: In function âvirt_to_head_pageâ:
include/linux/mm.h:254: warning: integer constant is too large for
âunsigned longâ type
include/linux/mm.h:254: warning: left shift count >= width of type
include/linux/mm.h:254: warning: integer constant is too large for
âunsigned longâ type
In file included from /home/matt/linux-iscsi/target/target/../include/
iscsi_linux_defs.h:27,
from /home/matt/linux-iscsi/target/target/../include/
iscsi_crc.h:21,
from /home/matt/linux-iscsi/target/target/../common/
iscsi_crc.c:35:
include/linux/mm.h:436:2: error: #error SECTIONS_WIDTH+NODES_WIDTH
+ZONES_WIDTH > FLAGS_RESERVED
In file included from /home/matt/linux-iscsi/target/target/../include/
iscsi_linux_defs.h:27,
from /home/matt/linux-iscsi/target/target/../include/
iscsi_crc.h:21,
from /home/matt/linux-iscsi/target/target/../common/
iscsi_crc.c:35:
include/linux/mm.h: In function âlowmem_page_addressâ:
include/linux/mm.h:538: warning: integer constant is too large for
âunsigned longâ type
include/linux/mm.h:538: warning: left shift count >= width of type
include/linux/mm.h:538: warning: integer constant is too large for
âunsigned longâ type
include/linux/mm.h:538: warning: cast to pointer from integer of
different size
In file included from include/linux/uaccess.h:5,
from include/linux/highmem.h:6,
from /home/matt/linux-iscsi/target/target/../include/
iscsi_linux_defs.h:36,
from /home/matt/linux-iscsi/target/target/../include/
iscsi_crc.h:21,
from /home/matt/linux-iscsi/target/target/../common/
iscsi_crc.c:35:
include/asm/uaccess.h: In function âcopy_from_userâ:
include/asm/uaccess.h:342: warning: integer constant is too large for
âunsigned longâ type
include/asm/uaccess.h:342: warning: integer constant is too large for
âunsigned longâ type
include/asm/uaccess.h:343: warning: integer constant is too large for
âunsigned longâ type
include/asm/uaccess.h:343: warning: integer constant is too large for
âunsigned longâ type
include/asm/uaccess.h: In function âcopy_to_userâ:
include/asm/uaccess.h:357: warning: integer constant is too large for
âunsigned longâ type
include/asm/uaccess.h:357: warning: integer constant is too large for
âunsigned longâ type
include/asm/uaccess.h:358: warning: integer constant is too large for
âunsigned longâ type
include/asm/uaccess.h:358: warning: integer constant is too large for
âunsigned longâ type
include/asm/uaccess.h: In function âclear_userâ:
include/asm/uaccess.h:452: warning: integer constant is too large for
âunsigned longâ type
include/asm/uaccess.h:452: warning: integer constant is too large for
âunsigned longâ type
include/asm/uaccess.h:453: warning: integer constant is too large for
âunsigned longâ type
include/asm/uaccess.h:453: warning: integer constant is too large for
âunsigned longâ type
make[3]: *** [/home/matt/linux-iscsi/target/target/../common/
iscsi_crc.o] Error 1
make[2]: *** [_module_/home/matt/linux-iscsi/target/target] Error 2
make[2]: Leaving directory `/home/matt/linux-cell-20080212-2.6.24'
make[1]: *** [all] Error 2
make[1]: Leaving directory `/home/matt/linux-iscsi/target/target'
make: *** [all] Error 2
xavia
problem for a few hours and had no solution. But then I happened to
read the Playstation3/iSCSI page one more time and noticed the comment
about putting ARCH=powerpc into the make command. Problem solved.
Guess I should have waited a little longer before sending that email.
Matt
n...@linux-iscsi.org
On Feb 12, 11:50 am, xavia <mleh...@gmail.com> wrote:
> I guess I wrote that email too soon. I had already been working on the
> problem for a few hours and had no solution. But then I happened to
> read the Playstation3/iSCSI page one more time and noticed the comment
> about putting ARCH=powerpc into the make command. Problem solved.
picked off the proper ARCH= from the kernel source for this particular
case.
I think it should change should be fine, and will make this happen
without the additional parameter.
Also, let me know if you have any issues getting your system up and
running.
--nab