run coreos on ms hyper-v

[Rimas Mocevicius]

Hi there,

Is any way I can install CoreOS VM on Microsoft Hyper-V?

I run there a few Ubuntu 12.04 VMs, but got sick of it's updates as these VMs are only to run docker containers 

CoreOS suits much better for me as it takes care itself of all system updates.

Do not want to use AWS for docker dev servers as we have to pay for it, but we have some spare computer

power on our local server, so the Hyper-V option for the CoreOS would be really nice to have 

Thanks

Rimas

[Brandon Philips]

On Wed, Mar 5, 2014 at 7:36 AM, Rimas Mocevicius <rmo...@gmail.com> wrote:
> Is any way I can install CoreOS VM on Microsoft Hyper-V?

We haven't tried Hyper-V yet but I imagine that the VMWare insecure
machine image would work fine:

http://coreos.com/docs/running-coreos/platforms/vmware/

Let us know how it goes; and if you encounter any problems. If you do
we can fix it up and work from there

Thanks!

Brandon

[Rimas Mocevicius]

Thanks Brandon for the tip.

I will give it a go and let you know in any case.

Thanks

Rimas

[Rimas Mocevicius]

Brandon,

I have tried to convert VMWare image to Hyper-V one, but had not much luck.

VM starts to boot and after few seconds screen gets unreadable I cannot see wants going on there anymore.

Checked our DHCP server, none IP address was allocated to that VM.

Also I see Vagrant 1.5 has Hyper-V support, would not be easy to add hyper-v support to coreos-vagrant one?

Thanks

Rimas

[lypanov]

I tried using CoreOS in Hyper-V via another path using the coreos-install script provided. It never seems to boot up and instead just restarts over and over.

It shows just "Starting new kernel" the screen then almost instantly turns black and the VM restarts itself.

Finally while writing this email I tried every option I could find and got it working - I increased the VM from 1GB to 2GB of RAM. Can't say I understand but

at least on Hyper-V this does the trick.

Hope it helps you also :)

Alex

[Alex Polvi]

It sounds like our kexec is causing the host to run out of memory. I wonder if there is some optimization we missed... because 1GB of memory seems like plenty to load a new kernel....

-Alex

[Rimas Mocevicius]

Alex,

Increasing the RAM to 2GB helped in my case too, I have converted VMWare image to Hyper-V one.

But still a console of the VM is unreadable when it boot ups.

Rimas

[lypanov]

On my system I've not seen the "unreadable console" issue but another thing to be wary of is having a legacy network adapter. I'm on Windows 8.1 BTW.

[lypanov]

1856M to be exact. 32M lower and it crashes... Very odd.

[Brandon Philips]

Hello Rimas-

On Sat, Mar 22, 2014 at 10:49 AM, Rimas Mocevicius <rmo...@gmail.com> wrote:
> Increasing the RAM to 2GB helped in my case too, I have converted VMWare
> image to Hyper-V one.
> But still a console of the VM is unreadable when it boot ups.

Would you mind writing some docs on how to do the conversion and get
CoreOS setup under Hyper-V? Our docs repository is here:
https://github.com/coreos/docs/tree/master/running-coreos/platforms

We will need to investigate the 2GB problem on Hyper-V. We boot under
KVM by default with 1024 MB of RAM. Until then it would be great to
document the 2GB issue.

Thanks!

Brandon

[Rimas Mocevicius]

Hi Brandon,

Sure, I will what I can do there on the documentation side.

But I can only do docs for the Hyper-V on Windows Server 2008 R2.

This the server we use at work. 

I do not use Windows myself as desktop at all.

It is a pity that Vagrant 1.5 supports only Hyper-V on Windows 8.1 :(

Rimas

[Rimas Mocevicius]

Hi Brandon,

I spent some time trying to run the latest CoreOS build on Windows 2008 R2 with the Hyper-V and no luck at all.

Converted VMWare image boots, screen get unreadable (everything is blur) and network IP doe snot get assigned too.

2GB trick does not help anymore

Also I tried to convert the vagrant VBox image to Hyper-V one, exactly the same issue as with above ones.

And of course I did not stop there, installed via "coreos-install" script, no luck with it too, the same problems as above.

So Windows 2008 R2 with the Hyper-V does not like CoreOS or via versa :)

I wish I can be more helpful there.

For now I will stick with the CoreOS cluster on my Mac.

Thanks

Rimas

[Greg KH]

On Fri, Apr 04, 2014 at 05:11:26AM -0700, Rimas Mocevicius wrote:
> Hi Brandon,
>

> I spent some time trying to run the latest CoreOS build on Windows 2008 R2 with
> the Hyper-V and no luck at all.
> Converted VMWare image boots, screen get unreadable (everything is blur) and
> network IP doe snot get assigned too.
> 2GB trick does not help anymore
>
> Also I tried to convert the vagrant VBox image to Hyper-V one, exactly the same
> issue as with above ones.
>
> And of course I did not stop there, installed via "coreos-install" script, no
> luck with it too, the same problems as above.
>
> So Windows 2008 R2 with the Hyper-V does not like CoreOS or via versa :)

There's a bug that the hyper-v developers have found with using kexec on
EFI platforms that I think you might be running into here. It should be
fixed in the kernel soon, and when that happens, we'll be able to pull
it into the coreos images as well.

Thanks for testing and letting us know about the problem,

greg k-h

[Rimas Mocevicius]

Cool, I can wait for the hyper-v fix

[Greg KH]

On Sat, Apr 05, 2014 at 05:16:34AM -0700, Rimas Mocevicius wrote:
> Cool, I can wait for the hyper-v fix

Great, as the developers are _still_ working on trying to get it right,
I don't know when it's going to be done, hopefully soon...

thanks,

greg k-h

[Rimas Mocevicius]

Thanks Greg for the update.

[Lucas Jans]

Hi Greg, I'm following this thread and eagerly waiting for a resolution. Is there any GitHub issue or pull request I can track?

---

Agency Revolution | Knowledge Base Articles | Navigator | Support Forum | Support Ticket | 541 330-2300 |

[Greg KH]

On Wed, May 28, 2014 at 07:06:53AM -0700, Lucas Jans wrote:
> Hi Greg, I'm following this thread and eagerly waiting for a resolution. Is
> there any GitHub issue or pull request I can track?

The Linux kernel doesn't use github or pull requests (they don't scale
at all), so sorry, there's nothing you can track here that I know of.

This is an upstream kernel kexec hyper-v issue, and I think the hyper-v
developers are still working on it. It might be fixed in the 3.15
kernel release to happen soon, but I haven't been paying attention to it
so I really don't know, sorry.

greg k-h

[Michael Marineau]

If 3.15 doesn't fix I'm going to just start shipping hyper-v images
that disable kexec, we already do this for all Xen (both PV and HVM)
images.

I'm assuming vhd is the preferred disk format that I should use for
those images. Does anyone know if there is a reasonable way to pass
cloud configs or at least ssh keys into the VM on hyper-v so we don't
have to bundle a known ssh key like the vmware_insecure images do?

[Alexander Kellett]

The trick I use locally with hyper v is to generate iso files with the user data file in and attach this to the vm before boot. Haven't tested this with azure yet as I'm scared about the kexec issue also being present on azure. I'll give it a shot now.

Trimis de pe iPhone-ul

[Alexander Kellett]

No ISO attachments on azure. I'll try it with a VHD attachment but will take a while.

Trimis de pe iPhone-ul meu de kkt

[Lucas Jans]

Hi Michael,

I think VHDX would be preferred, it performs faster than VHD, but is only compatible with Server 2012 and higher. 

There is a way for a Hyper-V host to mount a VHX/X to the host, at which point you can copy a file into it. The challenge you'll run into is first assigning a drive letter (you can script this) and writing to the file system. I'm not sure what file system CoreOS uses, but there are drivers available to read ext2/3/4 with Windows. http://www.ext2fsd.com/ Not sure if that would require a reboot to install, I'd be happy to test. Obviously some people would be sensitive to installing drivers on a production host. But I think the type of people who want to run CoreOS in their private Hyper-V data centers would be more flexible.

Mount-VHD

http://technet.microsoft.com/en-us/library/hh848551.aspx

Dismount-VHD

http://technet.microsoft.com/en-us/library/hh848562.aspx

[Alexander Kellett]

VHDX is not used by Azure and is difficult to generate from anything other than Windows IIRC. VHD is an “open” format and well supported by the VBox tooling for generation from Linux.

I’m going to give up on the Azure hosting side of this and stick to Hyper-V and then GCE for the actual deploys. Mostly because of my limits in upload bandwidth destroying any ability I have to actually test this..

Here is some more information I found that might be of help when creating VHDs:

   http://azure.microsoft.com/en-us/documentation/articles/virtual-machines-linux-create-upload-vhd/

My current progress was:

I tried creating a VHD with a exFAT filesystem with label config-2 but no luck, apparently the CoreOS image I’m using only has support for ext*, btrfs, and iso9660. I guess raw writing the iso9660 to the partition might work but I’ve no clue how to dd on Windows.

In case it helps, the Powershell to create init partition format and mount as a drive letter is:

New-VHD -Path test2.vhdx -SizeBytes 500MB | Mount-VHD -Passthru | Initialize-Disk -PassThru | New-Partition -AssignDriveLetter -UseMaximumSize | Format-Volume -FileSystem FAT32 -Confirm:$false -Force

I’d guessing that if CoreOS had a FAT driver it would have worked. Could also be that Initialize-Disk needs to be called with -PartitionStyle MBR as the default is GPT.

Cheers,

Alex

Trimis de la Windows Mail

De la: Lucas Jans
Trimis: ‎joi‎, ‎29‎ ‎mai‎ ‎2014 ‎3‎:‎07
Către: coreo...@googlegroups.com

[Michael Marineau]

On Thu, May 29, 2014 at 12:09 AM, Alexander Kellett <lyp...@live.com> wrote:
> VHDX is not used by Azure and is difficult to generate from anything other
> than Windows IIRC. VHD is an “open” format and well supported by the VBox
> tooling for generation from Linux.

Recent versions of qemu-img support writing VHDX but sounds like VHD
is the better default.

>
> I’m going to give up on the Azure hosting side of this and stick to Hyper-V
> and then GCE for the actual deploys. Mostly because of my limits in upload
> bandwidth destroying any ability I have to actually test this..
>
> Here is some more information I found that might be of help when creating
> VHDs:
>
> http://azure.microsoft.com/en-us/documentation/articles/virtual-machines-linux-create-upload-vhd/
>
> My current progress was:
>
> I tried creating a VHD with a exFAT filesystem with label config-2 but no
> luck, apparently the CoreOS image I’m using only has support for ext*,
> btrfs, and iso9660. I guess raw writing the iso9660 to the partition might
> work but I’ve no clue how to dd on Windows.

exFAT is not well supported on Linux but any of the normal FAT
variants work well. The config drive spec from OpenStack actually
dictates that the filesystem should be FAT or iso9660.

>
> In case it helps, the Powershell to create init partition format and mount
> as a drive letter is:
>
> New-VHD -Path test2.vhdx -SizeBytes 500MB | Mount-VHD -Passthru |
> Initialize-Disk -PassThru | New-Partition -AssignDriveLetter -UseMaximumSize
> | Format-Volume -FileSystem FAT32 -Confirm:$false -Force
>
> I’d guessing that if CoreOS had a FAT driver it would have worked. Could
> also be that Initialize-Disk needs to be called with -PartitionStyle MBR as
> the default is GPT.

Either GPT or MBR is fine. The disk image can actually be a plain
filesystem image without a partition table too which makes it easy to
use plain .iso files as disk images.

[Greg KH]

I talked to the hyper-v developers today, and this should all now be
fixed in Linux (the patches made it into 3.14.3), the main issue is that
Azure isn't running 2012 r2 which is required for this to work properly.
Azure will be rolling those changes out on their side soon, and is
asking me for a pointer to the CoreOS hyperv image so that they can test
that things now work properly.

Anyone have a pointer to the image that I can send to them for this
testing?

thanks,

greg k-h

[Michael Marineau]

I haven't started producing images specifically for hyperv yet but the
generic raw disk image should work:
http://alpha.release.core-os.net/amd64-usr/current/coreos_production_image.bin.bz2

[Greg KH]

Thanks, I'll point this to them and see what happens...

[Ed B]

For what it's worth, I can confirm that installing the raw disk image onto a 2012 R2 Hyper-V server (via coreos-install script from the iso) works, while doing the same on 2008 R2 and 2012 fails ("garbled" text, unusable, per the above).

-Ed

[John Montes]

I can attest that I've bumped into this issue with a Windows 2012 Standard Server :-)
After installing CoreOS to disk in a Hyper-V VM the console screen becomes blurry/unstable soon after rebooting, the VM doesnt appear on the network and cant be reached.

If we boot the VM to CoreOS via PXE boot and run in-memory we dont see the same issues and can reach the VM, but this is not an ideal setup :-)

Any estimates of when the fix would work on Windows 2012 Standard systems or I'm forced to upgrade to R2?

[Lucas Jans]

I'm facing the same issue. Hoping there's an easy fix for this.
--

------------------------------
*Agency Revolution <http://www.agencyrevolution.com/>* | Knowledge Base
Articles <https://support.agencyrevolution.com/home> | Navigator
<http://www.agencyrevolution.com/members/quantum-club-2.0> | Support Forum
<https://support.agencyrevolution.com/categories/20036027-Forums> | Support
Ticket <https://support.agencyrevolution.com/requests/anonymous/new> | 541
330-2300 |

[Rimas Mocevicius]

Hi guys,

I finally got our Win 2008 Server R2 updated to Win 2012 Server R2.

And decided to give another go for CoreOS there.

So I tried to install latest alpha/beta via coreos-install script from the iso install went fine and CoreOS boots fine without blury console out put anymore.

But this time encountered different problems:

1) I can not do reboot from CoreOS machine directly: most of the time it get stuck in reboot loop, so I have do "Turn Off" from Hyper-V manager

I can live with it for now.

2) Another more annoying problem is:

docker gives errors if I use fleet units to start services.

e.g. this unit file https://gist.github.com/rimusz/a70ff030adfef37e063d

shows these errors https://gist.github.com/rimusz/59cc58e30d7c27f14377

basically the main problem is this: Unable to find image '|' locally

But if I run directly via docker: /usr/bin/docker run --rm --name sshserver --hostname="$(hostname | cut -d. -f1)-sshserver" -p 2222:22 -v /media/data:/data myregistry:5000/sshserver:latest

it runs just fine.

Any tips how I can fix it?

Thanks

Rimas

[Rob Szumski]

When you run it manually, you’re running the command in a shell, which means your hostname subcommand is also executed. In the unit file, if you use ExecStart=/bin/sh -c “/usr/bin/docker run ….” it will execute in a shell and give you the same functionality.

 - Rob

[Rimas Mocevicius]

Thanks Rob,

I will try it out.

Rimas

[Rimas Mocevicius]

Hi Rob,

It did the trick, thanks.

But for curiosity why docker client needs to be run via shell on CoreOS hosted on hyper-v?

Thanks

Rimas

[Rob Szumski]

It’s not something specific to hyper-v or docker, it’s just that you’re trying to to run a subcommand inside of something that doesn’t support it. Running the command in a shell as described does support it.

[Rimas Mocevicius]

hmm, that's strange.

I usually run docker in fleet units on GCE instances as this:/usr/bin/docker run ..

and I had no problems with that.

On my hyper-v coreos one even /bin/bash -c "/usr/bin/docker run ..." does not work, only  "/bin/sh -c "does.

I do not think so I'm doing anything unsupported there.

Thanks

Rimas