Can the CoreOS support iSCSI?

[iamj...@gmail.com]

I boot up the coreos with pxe. but I can't mount the iSCSI to coreos.

 

Can you install it in the core.img?

 

thanks.

[Brandon Philips]

On Thu, Oct 10, 2013 at 1:51 AM, <iamj...@gmail.com> wrote:
> I boot up the coreos with pxe. but I can't mount the iSCSI to coreos.

We currently do not support iSCSI as a storage option. It may come
further down the road though.

I hope you can give it a shot with a regular disk or even just from
RAM with the PXE image.

Thanks,

Brandon

[Denis Jannot]

Hi Brandon,

Any news on the roadmap for iSCSI ?
In cloud scale datacenter, people try to use IP networks instead of SAN, so it would make sense to support iSCSI initiator on CoreOS

Thanks
Denis

[Greg KH]

On Wed, Jul 09, 2014 at 05:25:45AM -0700, Denis Jannot wrote:
> Hi Brandon,
>
> Any news on the roadmap for iSCSI ?
> In cloud scale datacenter, people try to use IP networks instead of SAN, so it
> would make sense to support iSCSI initiator on CoreOS

For iSCSI, what would be needed on a configuration/userspace side in
order to properly work for your environment? We can enable the kernel
code easily, it's just getting an easy-to-setup userspace configuration
is a bit more complex.

thanks,

greg k-h

[Brandon Philips]

On Wed, Jul 9, 2014 at 9:23 AM, Greg KH <gr...@kroah.com> wrote:
> For iSCSI, what would be needed on a configuration/userspace side in
> order to properly work for your environment? We can enable the kernel
> code easily, it's just getting an easy-to-setup userspace configuration
> is a bit more complex.

We have enabled all of the Kernel bits[1]. Now someone needs to build
containers that take advantage of it. I don't think there is a reason
that we need to ship the userspace bits directly in CoreOS.

Brandon

[1] https://github.com/coreos/coreos-overlay/blob/master/sys-kernel/coreos-kernel/files/x86_64_defconfig-3.15.2-r1#L1390

[Greg KH]

On Wed, Jul 09, 2014 at 09:24:57AM -0700, Brandon Philips wrote:
> On Wed, Jul 9, 2014 at 9:23 AM, Greg KH <gr...@kroah.com> wrote:
> > For iSCSI, what would be needed on a configuration/userspace side in
> > order to properly work for your environment? We can enable the kernel
> > code easily, it's just getting an easy-to-setup userspace configuration
> > is a bit more complex.
>
> We have enabled all of the Kernel bits[1]. Now someone needs to build
> containers that take advantage of it. I don't think there is a reason
> that we need to ship the userspace bits directly in CoreOS.

Ah, doh, I keep forgetting that a container could do all of the
configuration logic within itself, mount the data partition, and all
would be fine.

Jack, does that work for you?

thanks,

greg k-h

[Denis Jannot]

Hi Greg,

The idea would be to be able to use iSCSI at the host level and then to use docker run -v, but I can't install the iSCSI package on CoreOS.

The other option would be to use iSCSI at the container level and to share the volume with other containers, but I didn't find a way to make it working on CentOS or Ubuntu.

Best regards,

Denis

[Greg KH]

On Wed, Jul 09, 2014 at 09:12:53PM +0200, Denis Jannot wrote:
> Hi Greg,
>
> The idea would be to be able to use iSCSI at the host level and then to use
> docker run -v, but I can't install the iSCSI package on CoreOS.
> The other option would be to use iSCSI at the container level and to share the
> volume with other containers, but I didn't find a way to make it working on
> CentOS or Ubuntu.

What are you wanting to "host" on the iSCSI server? Your docker
containers? If so, yeah, this wouldn't work to put iSCSI in a
container.

I thought there was a way to share volumes between containers, but maybe
that is only if the "host" can see the volume...

What exact iscsi package are you looking to install?

thanks,

greg k-h

[Denis Jannot]

Greg,

I'd like to install the iSCSI initiator package to use a volume from an external disk array through iSCSI.

If I could install the iSCSI initiator in the CoreOS host, I could create a volume and then access it in a container.

If I could install the iSCSI initiator in a docker container, I could create the volume in the container and eventually share it with other containers.

In both cases, I could rediscover the iSCSI volume on another CoreOS host in case of host failure.

Best regards,
Denis

[iamjackhu]

Hi Greg k-h

Yes, it’s ok for me.
By now, many machine not only support start linux with PXE, but also iSCSI.

Thank
Jack

[Greg KH]

On Thu, Jul 10, 2014 at 08:52:14AM +0800, iamjackhu wrote:
> Hi Greg k-h
>
> Yes, it’s ok for me.
> By now, many machine not only support start linux with PXE, but also iSCSI.

Do you mean to put the root partition on iSCSI? If so, where does the
boot partition live?

thanks,

greg k-h

[Greg KH]

On Wed, Jul 09, 2014 at 09:30:58PM +0200, Denis Jannot wrote:
> Greg,
>
> I'd like to install the iSCSI initiator package to use a volume from an
> external disk array through iSCSI.
> If I could install the iSCSI initiator in the CoreOS host, I could create a
> volume and then access it in a container.
> If I could install the iSCSI initiator in a docker container, I could create
> the volume in the container and eventually share it with other containers.
> In both cases, I could rediscover the iSCSI volume on another CoreOS host in
> case of host failure.

Is there an open issue for this already? If not, can you make one and I
will take a look into it when I get the chance...

thanks,

greg k-h

[Michael Marineau]

We already have a PR but I haven't had time to dig into it much
myself, it is a complex looking beast.
https://github.com/coreos/coreos-overlay/pull/673

[Greg KH]

Ah, ok, thanks. The ebuilds don't look bad. The integration into
cloud-config, that's another story :)

greg k-h

[Daniel Ankers]

I've just had a go at accessing an iSCSI target from an initiator inside a Docker container running on CoreOS (to see if it was possible, not for any particular use case.)

This didn't work for me, it seems due to https://bugzilla.redhat.com/show_bug.cgi?id=1100000.

So at the moment that PR looks like it might be the only way to use iSCSI on CoreOS.

Dan

[Darren Shepherd]

That bug seems to point to the network namespace.  Did you trying --net=host?

Darren

[Daniel Ankers]

On 10 July 2014 17:31, Darren Shepherd <darren.s...@gmail.com> wrote:

That bug seems to point to the network namespace.  Did you trying --net=host?

I've just tried that successfully.  Thanks!

So the caveats for mounting iscsi from a container are:

1) The container must be run in privilege mode

2) The container needs to be run with --net=host

3) On my Centos container the iscsi drive came up as /dev/sda.  This did not exist in /dev on the container so I had to manually create the entries with mknod

This is obvious when you think about how Docker works, but it's worth mentioning that once the container was running iscsid and the drive was connected with iscsiadm -m node .... --login I was able to mount the drive in the coreos host, not just in the container running iscsid.

Dan

[Denis Jannot]

Hi Dan,

I've tried the following:
docker run -t -i --privileged --net=host centos /bin/sh
# yum install iscsi-initiator-utils
# service iscsid start
# iscsiadm -m discovery -t sendtargets -p 10.64.231.48

FATAL: Could not load /lib/modules/3.15.2+/modules.dep: No such file or directory
iscsiadm: iSCSI driver tcp is not loaded. Load the module then retry the command.

iscsiadm: Could not perform SendTargets discovery: iSCSI driver not found. Please make sure it is loaded, and retry the operation

What docker base image have you used ?

Thanks

Denis

[Denis Jannot]

Finally found that is works when adding -v /lib/modules:/lib/modules

[Daniel Ankers]

On 11 July 2014 13:35, Denis Jannot <dja...@gmail.com> wrote:

Finally found that is works when adding -v /lib/modules:/lib/modules

Hi Denis,

I did it a different way and did "modprobe iscsi_tcp" from CoreOS.

We probably ought to move any further talk of this onto coreos-user since the features required are obviously already in place.

Regards,

Dan

[Michael Marineau]

On Jul 11, 2014 5:45 AM, "Daniel Ankers" <md1...@md1clv.com> wrote:
>
> On 11 July 2014 13:35, Denis Jannot <dja...@gmail.com> wrote:
>>
>> Finally found that is works when adding -v /lib/modules:/lib/modules
>>
> Hi Denis,
> I did it a different way and did "modprobe iscsi_tcp" from CoreOS.

You can also add it to /etc/modules-load.d/something.conf

[Michael Marineau]

On Jul 11, 2014 2:21 AM, "Daniel Ankers" <md1...@md1clv.com> wrote:
>
> On 10 July 2014 17:31, Darren Shepherd <darren.s...@gmail.com> wrote:
>>
>> That bug seems to point to the network namespace.  Did you trying --net=host?
>
>
> I've just tried that successfully.  Thanks!
>
> So the caveats for mounting iscsi from a container are:
> 1) The container must be run in privilege mode
> 2) The container needs to be run with --net=host
> 3) On my Centos container the iscsi drive came up as /dev/sda.  This did not exist in /dev on the container so I had to manually create the entries with mknod

To avoid the mknod you can bind the hosts /dev into the container

[Denis Jannot]

Dan,

How have you created /dev/sda (will need in my case) ?

Thanks
Denis

[Daniel Ankers]

On 11 July 2014 16:51, Denis Jannot <dja...@gmail.com> wrote:

Dan,

How have you created /dev/sda (will need in my case) ?

Thanks
Denis

Hi Denis,

mknod /dev/sda  b 8 0

mknod /dev/sda1 b 8 1

mknod /dev/sda2 b 8 2

Regards,

Dan

[Denis Jannot]

Thanks